Most Linux machines are configured so that the kernel occupies the
upper 1G (0xc0000000 - 0xffffffff) of the 4G address space and
processes use the lower 3G (0x00000000 - 0xbfffffff). However, some
machine are configured with a 2G/2G split, with the kernel occupying
the upper 2G (0x80000000 - 0xffffffff) and processes using the lower
2G (0x00000000 - 0x7fffffff).
The problem
The prebuilt UML binaries on this site will not run on 2G/2G hosts
because UML occupies the upper .5G of the 3G process address space
(0xa0000000 - 0xbfffffff). Obviously, on 2G/2G hosts, this is right
in the middle of the kernel address space, so UML won't even load - it
will immediately segfault.
The solution
The fix for this is to rebuild UML from source after enabling
CONFIG_HOST_2G_2G (under 'General Setup'). This will cause UML to
load itself in the top .5G of that smaller process address space,
where it will run fine. See
this page if you need help building UML from source.
Hosted at
user-mode-linux-doc-20060501/COPYING 0000644 0000000 0000000 00000000655 12742461304 013503 0 ustar All documentation in this tree is Copyright (c) 2002 Jeff Dike.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1
or any later version published by the Free Software Foundation;
with the no invariant sections, with no Front-Cover Texts, and with the no
Back-Cover Texts.
A copy of the license is included in the file FDL in this directory.
user-mode-linux-doc-20060501/FDL 0000644 0000000 0000000 00000041022 12742461304 012771 0 ustar
Version 1.1, March 2000
Copyright (C) 2000 Free Software Foundation, Inc.
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
0. PREAMBLE
The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone
the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this
License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for
modifications made by others.
This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It
complements the GNU General Public License, which is a copyleft license designed for free software.
We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free
program should come with manuals providing the same freedoms that the software does. But this License is not limited to software
manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend
this License principally for works whose purpose is instruction or reference.
1. APPLICABILITY AND DEFINITIONS
This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under
the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is
addressed as "you".
A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with
modifications and/or translated into another language.
A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the
publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall
directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not
explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal,
commercial, philosophical, ethical or political position regarding them.
The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that
says that the Document is released under this License.
The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that
the Document is released under this License.
A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the
general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images
composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text
formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise
Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent.
A copy that is not "Transparent" is called "Opaque".
Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML
or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include
PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD
and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output
purposes only.
The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this
License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text
near the most prominent appearance of the work's title, preceding the beginning of the body of the text.
2. VERBATIM COPYING
You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the
copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no
other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further
copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large
enough number of copies you must also follow the conditions in section 3.
You may also lend copies, under the same conditions stated above, and you may publicly display copies.
3. COPYING IN QUANTITY
If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you
must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and
Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front
cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in
addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can
be treated as verbatim copying in other respects.
If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on
the actual cover, and continue the rest onto adjacent pages.
If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable
Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location
containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to
download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably
prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus
accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents
or retailers) of that edition to the public.
It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to
give them a chance to provide you with an updated version of the Document.
4. MODIFICATIONS
You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you
release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing
distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the
Modified Version:
A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions
(which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous
version if the original publisher of that version gives permission.
B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified
Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).
C. State on the Title page the name of the publisher of the Modified Version, as the publisher.
D. Preserve all the copyright notices of the Document.
E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.
F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under
the terms of this License, in the form shown in the Addendum below.
G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license
notice.
H. Include an unaltered copy of this License.
I. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and
publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one
stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified
Version as stated in the previous sentence.
J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and
likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History"
section. You may omit a network location for a work that was published at least four years before the Document itself, or if the
original publisher of the version it refers to gives permission.
K. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the
substance and tone of each of the contributor acknowledgements and/or dedications given therein.
L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent
are not considered part of the section titles.
M. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.
N. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.
If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material
copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the
list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.
You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various
parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of
a standard.
You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of
the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or
through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by
you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one,
on explicit permission from the previous publisher that added the old one.
The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or
imply endorsement of any Modified Version.
5. COMBINING DOCUMENTS
You may combine the Document with other documents released under this License, under the terms defined in section 4 above for
modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified,
and list them all as Invariant Sections of your combined work in its license notice.
The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single
copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by
adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number.
Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.
In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled
"History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all
sections entitled "Endorsements."
6. COLLECTIONS OF DOCUMENTS
You may make a collection consisting of the Document and other documents released under this License, and replace the individual
copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of
this License for verbatim copying of each of the documents in all other respects.
You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of
this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.
7. AGGREGATION WITH INDEPENDENT WORKS
A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a
storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is
claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained
works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the
Document.
If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of
the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate.
Otherwise they must appear on covers around the whole aggregate.
8. TRANSLATION
Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4.
Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations
of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this
License provided that you also include the original English version of this License. In case of a disagreement between the translation and
the original English version of this License, the original English version will prevail.
9. TERMINATION
You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt
to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However,
parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties
remain in full compliance.
10. FUTURE REVISIONS OF THIS LICENSE
The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new
versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See
http://www.gnu.org/copyleft/.
Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this
License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of
any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version
number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-1.html 0000644 0000000 0000000 00000013523 12742461304 017133 0 ustar
User Mode Linux HOWTO : IntroductionNext
Previous
Contents
User Mode Linux lets you run Linux inside itself! With that comes the
power to do all sorts of new things. It virtualises (or simulates, as
some people call it) Linux so that you can run an entire Linux where
once you would have only run a program.
You might have heard of functionality like this before. There are
quite a few projects whose goal is to nest operating systems in one
way or another: Linux on Linux, Windows on Linux, Linux on Windows,
Linux/s390 on Linux/anythingelse, and so on. Or even just x86 on
anything, where the 'x86' program can boot operating systems including
Linux.
Where x86 is involved there is the greatest concentration of
efforts. At the end of this HOWTO you'll find a list of alternative
projects. If all you want to do is run a copy of x86 Linux on another
copy of x86 Linux as fast as possible with little control then quite
possibly one of these other projects will so better than UML.
User Mode Linux (UML) is rather different from every other Linux
virtualisation project, either free or commercial. UML
strives to present itself as an ordinary program as much as
possible. Here are some of the outcomes of that philosophy:
Good speed with few compromises. UML compiles to
native machine code that runs just like any other compiled application
on the host. This makes it very much faster than portable
virtualisation schemes that implement an entire hardware architecture
in software. On the other hand, UML does not suffer from
the extreme hardware specificity of virtualisation systems that rely
on particular CPU features. UML runs applications inside
itself with normally at worst a 20% slowdown compared to the host
system, which modern hardware and clever system design can render
negligable in real terms.
Futureproof. Every time Linux gets improved so it can do
something new and clever that benefits the programs it runs, UML
automatically gets that facility. Software suspend,
fine-grained security control such as SE Linux, new filesystem
features, support for bigger/faster hardware... the same is not true
with those virtualisation systems that require major changes on the host
computer.
Flexible code. Normally an OS kernel is just that... a
kernel. It talks to hardware or maybe some virtualised hardware. But
UML can be viewed in many other ways. It would be possible
to turn it into a shared library, for example, so that other programs
could link to it to take advantage of things that Linux does very
well. It can be started as a subshell of an existing application. It
can use stin/stdout like any other program.
Portable. Really portable. UML has only just
started to be exploited for its portability, but there is promising
evidence that ports to x86 Windows, PowerPC Linux, x86 BSD and other
systems are very practical.
Mature. UML has been in development since 1999. One
indication of its robustness is that UML can be compiled to run within
UML, making it 'self-hosting'. Production systems are running on UML.
Free Software. UML is under the GPL (as it must be, being
part of the Linux kernel.)
Normally, the Linux Kernel talks straight to your hardware (video
card, keyboard, hard drives, etc), and any programs which run ask the
kernel to operate the hardware, like so:
+-----------+-----------+----+
| Process 1 | Process 2 | ...|
+-----------+-----------+----+
| Linux Kernel |
+----------------------------+
| Hardware |
+----------------------------+
The UML Kernel is different; instead of talking to the
hardware, it talks to a `real' Linux kernel (called the `host kernel'
from now on), like any other program. Programs can then run inside
User-Mode Linux as if they were running under a normal kernel, like
so:
+----------------+
| Process 2 | ...|
+-----------+----------------+
| Process 1 | User-Mode Linux|
+----------------------------+
| Linux Kernel |
+----------------------------+
| Hardware |
+----------------------------+
You can run gprof (profiling) and gcov (coverage testing).
You can play with your kernel without breaking things.
You can use it as a sandbox for testing new apps.
You can try new development kernels safely.
You can run different distributions simultaneously.
It's extremely fun.
Next
Previous
Contents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-10.html 0000644 0000000 0000000 00000025135 12742461304 017215 0 ustar
User Mode Linux HOWTO : The Management ConsoleNextPreviousContents
The UML management console is a low-level interface to the kernel,
somewhat like the i386 SysRq interface. Since there is a full-blown
operating system under UML, there is much greater flexibility possible
than with the SysRq mechanism.
There are a number of things you can do with the mconsole interface:
get the kernel version
add and remove devices
halt or reboot the machine
send SysRq commands
pause and resume the UML
make online backups without shutting down the UML
receive notifications of events of interest from within UML
monitor the internal state of the UML
You need the mconsole client (uml_mconsole) which is present in CVS
(/tools/mconsole) in 2.4.5-9um and later, and will be in the RPM in 2.4.6.
You also need CONFIG_MCONSOLE (under 'General Setup') enabled in UML.
When you boot UML, you'll see a line like:
mconsole initialized on /home/jdike/.uml/umlNJ32yL/mconsole
If you specify a unique machine id one the UML command line, i.e.
umid=debian
you'll see this
mconsole initialized on /home/jdike/.uml/debian/mconsole
That file is the socket that uml_mconsole will use to communicate
with UML. Run it with either the umid or the full path as its argument:
This takes no arguments. It prints the UML version.
(mconsole) version
OK Linux usermode 2.4.5-9um #1 Wed Jun 20 22:47:08 EDT 2001 i686
There are a couple actual uses for this. It's a simple no-op which
can be used to check that a UML is running. It's also a way of
sending an interrupt to the UML. This is sometimes useful on SMP
hosts, where there's a bug which causes signals to UML to be lost,
often causing it to appear to hang. Sending such a UML the mconsole
version command is a good way to 'wake it up' before networking has
been enabled, as it does not do anything to the function of the UML.
These take no arguments. They shut the machine down immediately, with
no syncing of disks and no clean shutdown of userspace. So, they are
pretty close to crashing the machine.
"config" adds a new device to the virtual machine or queries the configuration of an existing device.
Currently the ubd and network drivers support pulling devices. It takes one
argument, which is the device to add, with the same syntax as the
kernel command line.
(mconsole)
config ubd3=/home/jdike/incoming/roots/root_fs_debian22
OK
(mconsole) config eth1=mcast
OK
Querying the configuration of a device is handy when you don't know
before the boot what host device the UML device will attach to. This
is a problem with attaching consoles and serial lines to host pty or
pts devices. You have no way of knowing how to access them without
parsing the kernel messages. So, the syntax for this is the same as
above, except you don't specify a configuration
(mconsole) config ssl0
OK pty:/dev/ptyp0
(mconsole) config ubd0
OK /home/jdike/roots/cow.debian,/home/jdike/roots/debian_22
This is supported by the console, serial line, and ubd drivers. As
yet, the network drivers don't support this.
"remove" deletes a device from the system. Its argument is just the
name of the device to be removed. The device must be idle in
whatever sense the driver considers necessary. In the case of the ubd
driver, the removed block device must not be mounted, swapped on, or
otherwise open, and in the case of the network driver, the device must be down.
(mconsole) remove ubd3
OK
(mconsole) remove eth1
OK
This takes one argument, which is a single letter. It calls the
generic kernel's SysRq driver, which does whatever is called for by
that argument. See the SysRq documentation in Documentation/sysrq.txt
in your favorite kernel tree to see what letters are valid and what
they do.
This invokes the Ctl-Alt-Del action on init. What exactly this ends
up doing is up to /etc/inittab. Normally, it reboots the machine.
With UML, this is usually not desired, so if a halt would be better,
then find the section of inittab that looks like this
# What to do when CTRL-ALT-DEL is pressed.
ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
This puts the UML in a loop reading mconsole requests until a 'go'
mconsole command is recieved. This is very useful for making backups of
UML filesystems, as the UML can be stopped, then synced via 'sysrq s', so
that everything is written to the filesystem. You can then copy the
filesystem and then send the UML 'go' via mconsole.
Note that a UML running with more than one CPU will have problems after
you send the 'stop' command, as only one CPU will be held in a mconsole
loop and all others will continue as normal. This is a bug, and will
be fixed.
This resumes a UML after being paused by a 'stop' command. Note that
when the UML has resumed, TCP connections may have timed out and if the
UML is paused for a long period of time, crond might go a little crazy,
running all the jobs it didn't do earlier.
This takes a string as its argument, and will cause the UML to printk
the string so that it ends up in the kernel message log. This is
intended for use in honeypots by allowing the UML-specific stuff in
the kernel log to be replaced with messages that don't expose the
machine as being a UML.
It is possible to make a backup of a UML's data without shutting it
down. The idea is to pause it, make it flush out its data, copy the
filesystem to a safe place, and then resume it. This should usually
take seconds, while shutting down and rebooting the UML could take
minutes. The exact procedure is this:
(mconsole) stop
(mconsole) sysrq s
host% # Copy the UML's filesystem someplace safe
(mconsole) go
By causing UML to flush its data out to disk, the 'sysrq s' will cause
the filesystem to be a clean image. Of course, no guarantees are made
for process data which hadn't been written back to the kernel, but the
filesystem itself won't need an fsck if it's booted.
The mconsole interface also provides a mechanism for processes inside
a UML to send messages to an mconsole client on the host. The
procedure is this:
Create a unix socket and pass that to UML on the command line as the
mconsole notification socket
mconsole=notify:<bf>socket
A /proc/mconsole file will be created inside UML
Anything that is written to it will be turned into an mconsole
notification which your mconsole client should be listening for on the
notification socket
A common use for this mechanism is to have an rc script inside UML
send a message out that the UML has booted to a certain stage, and
that something on the host which depends on that can proceed.
However, this is a completely general mechanism which can be used to
communicate any information at all to the host.
There is a demo mconsole notification client in the utilities tarball
in mconsole/notify.pl. This is only a demo, and as such, isn't very
useful by itself. It should be customized to fit into whatever
environment you are setting up.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-11.html 0000644 0000000 0000000 00000034154 12742461304 017217 0 ustar
User Mode Linux HOWTO : Kernel debuggingNextPreviousContents
This page describes kernel debugging with UML running in tt mode (go
here for the details on
skas and tt mode). Kernel debugging in skas mode is described
here .
Since the UML runs as a normal Linux process, it is
possible to debug it with gdb almost like any other process. It is
slightly different because the kernel's threads are already being
ptraced for system call interception, so gdb can't ptrace them.
However, a mechanism has been added to work around that problem.
In order to debug the kernel, you need build it from source. See
Compiling the kernel and modules for
information on doing that. Make sure that you enable CONFIG_DEBUGSYM
and CONFIG_PT_PROXY during the config. These will compile the kernel
with -g, and enable the ptrace proxy so that gdb works with UML,
respectively.
You can have the kernel running under the control of gdb from the
beginning by putting 'debug' on the command line. You will get an
xterm with gdb running inside it. The kernel will send some commands
to gdb which will leave it stopped at the beginning of start_kernel.
At this point, you can get things going with 'next', 'step', or 'cont'.
There is a transcript of a debugging session
here , with breakpoints being set in the
scheduler and in an interrupt handler.
Not every bug is evident in the currently running process. Sometimes,
processes hang in the kernel when they shouldn't because they've
deadlocked on a semaphore or something similar. In this case, when
you ^C gdb and get a backtrace, you will see the idle thread, which
isn't very relevant.
What you want is the stack of whatever process is sleeping when it
shouldn't be. You need to figure out which process that is, which is
generally fairly easy. Then you need to get its host process id,
which you can do either by looking at ps on the host or at
task.thread.extern_pid in gdb.
Now what you do is this:
detach from the current thread
(UML gdb) det
attach to the thread you are interested in
(UML gdb) att <host pid>
look at its stack and anything else of interest
(UML gdb) bt
Note that you can't do anything at this point that requires that a
process execute, e.g. calling a function
when you're done looking at that process, reattach to the current thread
and continue it
(UML gdb)
att 1
(UML gdb)
c
Here, specifying any pid which is not the process id of a UML thread
will cause gdb to reattach to the current thread. I commonly use 1,
but any other invalid pid would work.
gdb has support for debugging code which is dynamically loaded into
the process. This support is what is needed to debug kernel modules
under UML.
Using that support is somewhat complicated. You have to tell gdb what
object file you just loaded into UML and where in memory it is. Then,
it can read the symbol table, and figure out where all the symbols are
from the load address that you provided. It gets more interesting
when you load the module again (i.e. after an rmmod). You have to
tell gdb to forget about all its symbols, including the main UML ones
for some reason, then load then all back in again.
There's an easy way and a hard way to do this. The easy way is to use
the umlgdb expect script written by Chandan Kudige. It basically
automates the process for you.
First, you must tell it where your modules are. There is a list in
the script that looks like this:
set MODULE_PATHS {
"fat" "/usr/src/uml/linux-2.4.18/fs/fat/fat.o"
"isofs" "/usr/src/uml/linux-2.4.18/fs/isofs/isofs.o"
"minix" "/usr/src/uml/linux-2.4.18/fs/minix/minix.o"
}
You change that to list the names and paths of the modules that you
are going to debug. Then you run it from the toplevel directory of
your UML pool and it basically tells you what to do:
******** GDB pid is 21903 ********
Start UML as: ./linux <kernel switches> debug gdb-pid=21903
GNU gdb 5.0rh-5 Red Hat Linux 7.1
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) b sys_init_module
Breakpoint 1 at 0xa0011923: file module.c, line 349.
(gdb) att 1
After you run UML and it sits there doing nothing, you hit return at
the 'att 1' and continue it:
Attaching to program: /home/jdike/linux/2.4/um/./linux, process 1
0xa00f4221 in __kill ()
(UML gdb) c
Continuing.
At this point, you debug normally. When you insmod something, the
expect magic will kick in and you'll see something like:
*** Module hostfs loaded ***
Breakpoint 1, sys_init_module (name_user=0x805abb0 "hostfs",
mod_user=0x8070e00) at module.c:349
349 char *name, *n_name, *name_tmp = NULL;
(UML gdb) finish
Run till exit from #0 sys_init_module (name_user=0x805abb0 "hostfs",
mod_user=0x8070e00) at module.c:349
0xa00e2e23 in execute_syscall (r=0xa8140284) at syscall_kern.c:411
411 else res = EXECUTE_SYSCALL(syscall, regs);
Value returned is $1 = 0
(UML gdb)
p/x (int)module_list + module_list->size_of_struct
$2 = 0xa9021054
(UML gdb) symbol-file ./linux
Load new symbol table from "./linux"? (y or n) y
Reading symbols from ./linux...
done.
(UML gdb)
add-symbol-file /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o 0xa9021054
add symbol table from file "/home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o" at
.text_addr = 0xa9021054
(y or n) y
Reading symbols from /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o...
done.
(UML gdb) p *module_list
$1 = {size_of_struct = 84, next = 0xa0178720, name = 0xa9022de0 "hostfs",
size = 9016, uc = {usecount = {counter = 0}, pad = 0}, flags = 1,
nsyms = 57, ndeps = 0, syms = 0xa9023170, deps = 0x0, refs = 0x0,
init = 0xa90221f0 <init_hostfs>, cleanup = 0xa902222c <exit_hostfs>,
ex_table_start = 0x0, ex_table_end = 0x0, persist_start = 0x0,
persist_end = 0x0, can_unload = 0, runsize = 0, kallsyms_start = 0x0,
kallsyms_end = 0x0,
archdata_start = 0x1b855 <Address 0x1b855 out of bounds>,
archdata_end = 0xe5890000 <Address 0xe5890000 out of bounds>,
kernel_data = 0xf689c35d <Address 0xf689c35d out of bounds>}
>> Finished loading symbols for hostfs ...
That's the easy way. It's highly recommended. The umlgdb script is
available in the
UML utilities tarball in tools/umlgdb/umlgdb. The
hard way is described below in case you're interested in what's going
on.
Boot the kernel under the debugger and load the module with insmod or
modprobe. With gdb, do:
(UML gdb) p module_list
This is a list of modules that have been loaded into the kernel, with
the most recently loaded module first. Normally, the module you want
is at module_list. If it's not, walk down the next links, looking at
the name fields until find the module you want to debug. Take the
address of that structure, and add module.size_of_struct (which in
2.4.10 kernels is 96 (0x60)) to it. Gdb can make this hard addition for
you :-):
The offset from the module start occasionally changes (before 2.4.0, it
was module.size_of_struct + 4), so it's a good idea to check the init and
cleanup addresses once in a while, as describe below.
Now do:
Tell gdb you really want to do it, and you're in business.
If there's any doubt that you got the offset right, like breakpoints
appear not to work, or they're appearing in the wrong place, you can
check it by looking at the module structure. The init and cleanup
fields should look like:
with no offsets on the symbol names. If the names are right, but they
are offset, then the offset tells you how much you need to add to the
address you gave to add-symbol-file.
When you want to load in a new version of the module, you need to get
gdb to forget about the old one. The only way I've found to do that
is to tell gdb to forget about all symbols that it knows about:
(UML gdb) symbol-file
Then reload the symbols from the kernel binary:
(UML gdb) symbol-file /path/to/kernel
and repeat the process above. You'll also need to re-enable breakpoints.
They were disabled when you dumped all the symbols because gdb
couldn't figure out where they should go.
If you don't have the kernel running under gdb, you can attach gdb to
it later by sending the tracing thread a SIGUSR1. The first line of
the console output identifies its pid:
tracing thread pid = 20093
When you send it the signal:
host% kill -USR1 20093
you will get an xterm with gdb running in it.
If you have the mconsole compiled into UML, then the mconsole client
can be used to start gdb:
UML has support for attaching to an already running debugger rather than
starting gdb itself. This is present in CVS as of
17 Apr 2001. I sent it to Alan for inclusion in
the ac tree, and it will be in my 2.4.4 release.
This is useful when gdb is a subprocess of some UI, such as emacs or ddd. It
can also be used to run debuggers other than gdb on UML. Below is an example
of using strace as an alternate debugger.
To do this, you need to get the pid of the debugger and pass it in with the
'gdb-pid=<pid>' switch along with the 'debug' switch.
If you are using gdb under some UI, then tell it to 'att 1', and you'll find
yourself attached to UML.
If you are using something other than gdb as your debugger, then you'll need
to get it to do the equivalent of 'att 1' if it doesn't do it automatically.
An example of an alternate debugger is strace. You can strace the actual
kernel as follows:
Run the following in a shell
host%
sh -c 'echo pid=$$; echo -n hit return; read x; exec strace -p 1 -o strace.out'
Run UML with 'debug' and 'gdb-pid=<pid>' with the pid printed out by the
previous command
Hit return in the shell, and UML will start running, and strace output will
start accumulating in the output file.
Note that this is different from running
host% strace ./linux
That will strace only the main UML thread, the tracing thread, which doesn't
do any of the actual kernel work. It just oversees the virtual machine. In
contrast, using strace as described above will show you the low-level activity
of the virtual machine.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-12.html 0000644 0000000 0000000 00000065700 12742461304 017221 0 ustar
User Mode Linux HOWTO : Kernel debugging examplesNextPreviousContents
When booting up the kernel, fsck failed, and dropped me into a shell to fix
things up. I ran fsck -y, which hung:
Setting hostname uml [ OK ]
Checking root filesystem
/dev/fhd0 was not cleanly unmounted, check forced.
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.
/dev/fhd0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
[ FAILED ]
*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell.
Give root password for maintenance
(or type Control-D for normal startup):
[root@uml /root]# fsck -y /dev/fhd0
fsck -y /dev/fhd0
Parallelizing fsck version 1.14 (9-Jan-1999)
e2fsck 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
/dev/fhd0 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780. Ignore error? yes
Inode 19780, i_blocks is 1548, should be 540. Fix? yes
Pass 2: Checking directory structure
Error reading block 49405 (Attempt to read block from filesystem resulted in short read). Ignore error? yes
Directory inode 11858, block 0, offset 0: directory corrupted
Salvage? yes
Missing '.' in directory inode 11858.
Fix? yes
Missing '..' in directory inode 11858.
Fix? yes
The standard drill in this sort of situation is to fire up gdb on the
signal thread, which, in this case, was pid 1935. In another window, I
run gdb and attach pid 1935.
~/linux/2.3.26/um 1016: gdb linux
GNU gdb 4.17.0.11 with Linux support
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) att 1935
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1935
0x100756d9 in __wait4 ()
Let's see what's currently running:
(gdb) p current_task.pid
$1 = 0
It's the idle thread, which means that fsck went to sleep for some
reason and never woke up.
Let's guess that the last process in the process list is fsck:
(gdb) p current_task.prev_task.comm
$13 = "fsck.ext2\000\000\000\000\000\000"
The interesting things here are the fact that its .thread.syscall.id
is __NR_write (see the big switch in arch/um/kernel/syscall_kern.c or the
defines in include/asm-um/arch/unistd.h), and that it never returned.
Also, its .request.op is OP_SWITCH (see arch/um/include/user_util.h).
These mean that it went into a write, and, for some reason, called
schedule().
The fact that it never returned from write means that its stack
should be fairly interesting. Its pid is 1980 (.thread.extern_pid).
That process is being ptraced by the signal thread, so it must be
detached before gdb can attach it:
(gdb) call detach(1980)
Program received signal SIGSEGV, Segmentation fault.
<function called from gdb>
The program being debugged stopped while in a function called from GDB.
When the function (detach) is done executing, GDB will silently
stop (instead of continuing to evaluate the expression containing
the function call).
(gdb) call detach(1980)
$15 = 0
The first detach segfaults for some reason, and the second one
succeeds.
Now I detach from the signal thread, attach to the fsck thread, and
look at its stack:
(gdb) det
Detaching from program: /home/dike/linux/2.3.26/um/linux Pid 1935
(gdb) att 1980
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1980
0x10070451 in __kill ()
(gdb) bt
#0 0x10070451 in __kill ()
#1 0x10068ccd in usr1_pid (pid=1980) at process.c:30
#2 0x1006a03f in _switch_to (prev=0x50072000, next=0x507e8000)
at process_kern.c:156
#3 0x1006a052 in switch_to (prev=0x50072000, next=0x507e8000, last=0x50072000)
at process_kern.c:161
#4 0x10001d12 in schedule () at sched.c:777
#5 0x1006a744 in __down (sem=0x507d241c) at semaphore.c:71
#6 0x1006aa10 in __down_failed () at semaphore.c:157
#7 0x1006c5d8 in segv_handler (sc=0x5006e940) at trap_user.c:174
#8 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
#9 <signal handler called>
#10 0x10155404 in errno ()
#11 0x1006c0aa in segv (address=1342179328, is_write=2) at trap_kern.c:50
#12 0x1006c5d8 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
#13 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
#14 <signal handler called>
#15 0xc0fd in ?? ()
#16 0x10016647 in sys_write (fd=3,
buf=0x80b8800 <Address 0x80b8800 out of bounds>, count=1024)
at read_write.c:159
#17 0x1006d5b3 in execute_syscall (syscall=4, args=0x5006ef08)
at syscall_kern.c:254
#18 0x1006af87 in really_do_syscall (sig=12) at syscall_user.c:35
#19 <signal handler called>
#20 0x400dc8b0 in ?? ()
The interesting things here are :
There are two segfaults on this stack (frames 9 and 14)
The first faulting address (frame 11) is 0x50000800
(gdb) p (void *)1342179328
$16 = (void *) 0x50000800
The initial faulting address is interesting because it is on the idle
thread's stack. I had been seeing the idle thread segfault for no
apparent reason, and the cause looked like stack corruption. In hopes
of catching the culprit in the act, I had turned off all protections
to that stack while the idle thread wasn't running. This apparently
tripped that trap.
However, the more immediate problem is that second segfault and I'm
going to concentrate on that. First, I want to see where the fault
happened, so I have to go look at the sigcontent struct in frame 8:
(gdb) up
#1 0x10068ccd in usr1_pid (pid=1980) at process.c:30
30 kill(pid, SIGUSR1);
(gdb)
#2 0x1006a03f in _switch_to (prev=0x50072000, next=0x507e8000)
at process_kern.c:156
156 usr1_pid(getpid());
(gdb)
#3 0x1006a052 in switch_to (prev=0x50072000, next=0x507e8000, last=0x50072000)
at process_kern.c:161
161 _switch_to(prev, next);
(gdb)
#4 0x10001d12 in schedule () at sched.c:777
777 switch_to(prev, next, prev);
(gdb)
#5 0x1006a744 in __down (sem=0x507d241c) at semaphore.c:71
71 schedule();
(gdb)
#6 0x1006aa10 in __down_failed () at semaphore.c:157
157 }
(gdb)
#7 0x1006c5d8 in segv_handler (sc=0x5006e940) at trap_user.c:174
174 segv(sc->cr2, sc->err & 2);
(gdb)
#8 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
182 segv_handler(sc);
(gdb) p *sc
Cannot access memory at address 0x0.
That's not very useful, so I'll try a more manual method:
(gdb) p (void *)268480945
$20 = (void *) 0x1000b1b1
(gdb) i sym $20
handle_mm_fault + 57 in section .text
Specifically, it's in pte_alloc:
(gdb) i line *$20
Line 124 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1b1 <handle_mm_fault+57>
and ends at 0x1000b1b7 <handle_mm_fault+63>.
To find where in handle_mm_fault this is, I'll jump forward in the
code until I see an address in that procedure:
(gdb) i line *0x1000b1c0
Line 126 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1b7 <handle_mm_fault+63>
and ends at 0x1000b1c3 <handle_mm_fault+75>.
(gdb) i line *0x1000b1d0
Line 131 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1d0 <handle_mm_fault+88>
and ends at 0x1000b1da <handle_mm_fault+98>.
(gdb) i line *0x1000b1e0
Line 61 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1da <handle_mm_fault+98>
and ends at 0x1000b1e1 <handle_mm_fault+105>.
(gdb) i line *0x1000b1f0
Line 134 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1f0 <handle_mm_fault+120>
and ends at 0x1000b200 <handle_mm_fault+136>.
(gdb) i line *0x1000b200
Line 135 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b200 <handle_mm_fault+136>
and ends at 0x1000b208 <handle_mm_fault+144>.
(gdb) i line *0x1000b210
Line 139 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b210 <handle_mm_fault+152>
and ends at 0x1000b219 <handle_mm_fault+161>.
(gdb) i line *0x1000b220
Line 1168 of "memory.c" starts at address 0x1000b21e <handle_mm_fault+166>
and ends at 0x1000b222 <handle_mm_fault+170>.
Something is apparently wrong with the page tables or vma_structs,
so lets go back to frame 11 and have a look at them:
This also pretty bogus. With all of the 0x80xxxxx and 0xaffffxxx
addresses, this is looking like a stack was plonked down on top of
these structures. Maybe it's a stack overflow from the next page:
(gdb) p vma
$25 = (struct vm_area_struct *) 0x507d2434
That's towards the lower quarter of the page, so that would have to
have been pretty heavy stack overflow:
It's not stack overflow. The only "stack-like" piece of this data
is the vma_struct itself.
At this point, I don't see any avenues to pursue, so I just have to
admit that I have no idea what's going on. What I will do, though, is
stick a trap on the segfault handler which will stop if it sees any
writes to the idle thread's stack. That was the thing that happened
first, and it may be that if I can catch it immediately, what's going
on will be somewhat clearer.
After setting a trap in the SEGV handler for accesses to the signal
thread's stack, I reran the kernel.
fsck hung again, this time by hitting the trap:
Setting hostname uml [ OK ]
Checking root filesystem
/dev/fhd0 contains a file system with errors, check forced.
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.
/dev/fhd0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
[ FAILED ]
*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell.
Give root password for maintenance
(or type Control-D for normal startup):
[root@uml /root]# fsck -y /dev/fhd0
fsck -y /dev/fhd0
Parallelizing fsck version 1.14 (9-Jan-1999)
e2fsck 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
/dev/fhd0 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780. Ignore error? yes
Pass 2: Checking directory structure
Error reading block 49405 (Attempt to read block from filesystem resulted in short read). Ignore error? yes
Directory inode 11858, block 0, offset 0: directory corrupted
Salvage? yes
Missing '.' in directory inode 11858.
Fix? yes
Missing '..' in directory inode 11858.
Fix? yes
Untested (4127) [100fe44c]: trap_kern.c line 31
I need to get the signal thread to detach from pid 4127 so that I can
attach to it with gdb. This is done by sending it a SIGUSR1, which is
caught by the signal thread, which detaches the process:
kill -USR1 4127
Now I can run gdb on it:
~/linux/2.3.26/um 1034: gdb linux
GNU gdb 4.17.0.11 with Linux support
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) att 4127
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 4127
0x10075891 in __libc_nanosleep ()
The backtrace shows that it was in a write and that the fault
address (address in frame 3) is 0x50000800, which is right in the
middle of the signal thread's stack page:
(gdb) bt
#0 0x10075891 in __libc_nanosleep ()
#1 0x1007584d in __sleep (seconds=1000000)
at ../sysdeps/unix/sysv/linux/sleep.c:78
#2 0x1006ce9a in stop () at user_util.c:191
#3 0x1006bf88 in segv (address=1342179328, is_write=2) at trap_kern.c:31
#4 0x1006c628 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
#5 0x1006c63c in kern_segv_handler (sig=11) at trap_user.c:182
#6 <signal handler called>
#7 0xc0fd in ?? ()
#8 0x10016647 in sys_write (fd=3, buf=0x80b8800 "R.", count=1024)
at read_write.c:159
#9 0x1006d603 in execute_syscall (syscall=4, args=0x5006ef08)
at syscall_kern.c:254
#10 0x1006af87 in really_do_syscall (sig=12) at syscall_user.c:35
#11 <signal handler called>
#12 0x400dc8b0 in ?? ()
#13 <signal handler called>
#14 0x400dc8b0 in ?? ()
#15 0x80545fd in ?? ()
#16 0x804daae in ?? ()
#17 0x8054334 in ?? ()
#18 0x804d23e in ?? ()
#19 0x8049632 in ?? ()
#20 0x80491d2 in ?? ()
#21 0x80596b5 in ?? ()
(gdb) p (void *)1342179328
$3 = (void *) 0x50000800
Going up the stack to the segv_handler frame and looking at where
in the code the access happened shows that it happened near line 110
of block_dev.c:
(gdb) up
#1 0x1007584d in __sleep (seconds=1000000)
at ../sysdeps/unix/sysv/linux/sleep.c:78
../sysdeps/unix/sysv/linux/sleep.c:78: No such file or directory.
(gdb)
#2 0x1006ce9a in stop () at user_util.c:191
191 while(1) sleep(1000000);
(gdb)
#3 0x1006bf88 in segv (address=1342179328, is_write=2) at trap_kern.c:31
31 KERN_UNTESTED();
(gdb)
#4 0x1006c628 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
174 segv(sc->cr2, sc->err & 2);
(gdb) p *sc
$1 = {gs = 0, __gsh = 0, fs = 0, __fsh = 0, es = 43, __esh = 0, ds = 43,
__dsh = 0, edi = 1342179328, esi = 134973440, ebp = 1342631484,
esp = 1342630864, ebx = 256, edx = 0, ecx = 256, eax = 1024, trapno = 14,
err = 6, eip = 268550834, cs = 35, __csh = 0, eflags = 66070,
esp_at_signal = 1342630864, ss = 43, __ssh = 0, fpstate = 0x0, oldmask = 0,
cr2 = 1342179328}
(gdb) p (void *)268550834
$2 = (void *) 0x1001c2b2
(gdb) i sym $2
block_write + 1090 in section .text
(gdb) i line *$2
Line 209 of "/home/dike/linux/2.3.26/um/include/asm/arch/string.h"
starts at address 0x1001c2a1 <block_write+1073>
and ends at 0x1001c2bf <block_write+1103>.
(gdb) i line *0x1001c2c0
Line 110 of "block_dev.c" starts at address 0x1001c2bf <block_write+1103>
and ends at 0x1001c2e3 <block_write+1139>.
Looking at the source shows that the fault happened during a
call to copy_to_user to copy the data into the kernel:
p is the pointer which must contain 0x50000800, since buf contains
0x80b8800 (frame 8 above). It is defined as:
p = offset + bh->b_data;
I need to figure out what bh is, and it just so happens that bh is
passed as an argument to mark_buffer_uptodate and mark_buffer_dirty a
few lines later, so I do a little disassembly:
(gdb) disas 0x1001c2bf 0x1001c2e0
Dump of assembler code from 0x1001c2bf to 0x1001c2d0:
0x1001c2bf <block_write+1103>: addl %eax,0xc(%ebp)
0x1001c2c2 <block_write+1106>: movl 0xfffffdd4(%ebp),%edx
0x1001c2c8 <block_write+1112>: btsl $0x0,0x18(%edx)
0x1001c2cd <block_write+1117>: btsl $0x1,0x18(%edx)
0x1001c2d2 <block_write+1122>: sbbl %ecx,%ecx
0x1001c2d4 <block_write+1124>: testl %ecx,%ecx
0x1001c2d6 <block_write+1126>: jne 0x1001c2e3 <block_write+1139>
0x1001c2d8 <block_write+1128>: pushl $0x0
0x1001c2da <block_write+1130>: pushl %edx
0x1001c2db <block_write+1131>: call 0x1001819c <__mark_buffer_dirty>
End of assembler dump.
At that point, bh is in %edx (address 0x1001c2da), which is
calculated at 0x1001c2c2 as %ebp + 0xfffffdd4, so I figure exactly
what that is, taking %ebp from the sigcontext_struct above:
(gdb) p (void *)1342631484
$5 = (void *) 0x5006ee3c
(gdb) p 0x5006ee3c+0xfffffdd4
$6 = 1342630928
(gdb) p (void *)$6
$7 = (void *) 0x5006ec10
(gdb) p *((void **)$7)
$8 = (void *) 0x50100200
Now, I look at the structure to see what's in it, and particularly,
what its b_data field contains:
The b_data field is indeed 0x50000800, so the question becomes how
that happened. The rest of the structure looks fine, so this probably is
not a case of data corruption. It happened on purpose somehow.
The b_page field is a pointer to the page_struct representing the
0x50000000 page. Looking at it shows the kernel's idea of the state
of that page:
Some sanity-checking: the virtual field shows the "virtual" address
of this page, which in this kernel is the same as its "physical"
address, and the page_struct itself should be mem_map[0], since it
represents the first page of memory:
(gdb) p (void *)1342177280
$18 = (void *) 0x50000000
(gdb) p mem_map
$19 = (mem_map_t *) 0x50004000
These check out fine.
Now to check out the page_struct itself. In particular, the flags
field shows whether the page is considered free or not:
(gdb) p (void *)132
$21 = (void *) 0x84
The "reserved" bit is the high bit, which is definitely not set, so
the kernel considers the signal stack page to be free and available to
be used.
At this point, I jump to conclusions and start looking at my early
boot code, because that's where that page is supposed to be reserved.
In my setup_arch procedure, I have the following code which looks
just fine:
Two stack pages have already been allocated, and low_physmem points
to the third page, which is the beginning of free memory.
The init_bootmem call declares the entire memory to the boot memory
manager, which marks it all reserved. The free_bootmem call frees up
all of it, except for the first two pages. This looks correct to me.
So, I decide to see init_bootmem run and make sure that it is
marking those first two pages as reserved. I never get that far.
Stepping into init_bootmem, and looking at bootmem_map before
looking at what it contains shows the following:
(gdb) p bootmem_map
$3 = (void *) 0x50000000
Aha! The light dawns. That first page is doing double duty as a
stack and as the boot memory map. The last thing that the boot memory
manager does is to free the pages used by its memory map, so this page is
getting freed even its marked as reserved.
The fix was to initialize the boot memory manager before allocating
those two stack pages, and then allocate them through the boot memory
manager. After doing this, and fixing a couple of subsequent buglets,
the stack corruption problem disappeared.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-13.html 0000644 0000000 0000000 00000033430 12742461304 017215 0 ustar
User Mode Linux HOWTO : What to do when UML doesn't workNextPreviousContents
As of test11, it is necessary to have "ARCH=um" in the environment or
on the make command line for all steps in building UML, including
clean, distclean, or mrproper, config, menuconfig, or xconfig, dep,
and linux. If you forget for any of them, the i386 build seems to
contaminate the UML build. If this happens, start from scratch with
host%
make mrproper ARCH=um
and repeat the build process with ARCH=um on all the steps.
Another cause of strange compilation errors is building UML in
/usr/src/linux. If you do this, the first thing you need to do is
clean up the mess you made. The /usr/src/linux/asm link will now
point to /usr/src/linux/asm-um. Make it point back to
/usr/src/linux/asm-i386. Then, move your UML pool someplace else and
build it there. Also see below, where a more specific set of symptoms
is described.
VFS: Mounted root (ext2 filesystem) readonly.
Mounted devfs on /dev
You're probably running a recent distribution on an old machine. I
saw this with the RH7.1 filesystem running on a Pentium. The shared
library loader, ld.so, was executing an instruction (cmove) which the
Pentium didn't support. That instruction was apparently added later.
If you run UML under the debugger, you'll see the hang caused by one
instruction causing an infinite SIGILL stream.
I saw this on reiserfs 3.5.21 and it seems to be fixed in 3.5.27.
Panics preceded by
Detaching pid nnnn
are
diagnostic of this problem. This is a reiserfs bug which causes a
thread to occasionally read stale data from a mmapped page shared with
another thread. The fix is to upgrade the filesystem or to have /tmp be
an ext2 filesystem.
This happens when you build in /usr/src/linux. The UML build makes
the include/asm link point to include/asm-um. /usr/include/asm points
to /usr/src/linux/include/asm, so when that link gets moved, files
which need to include the asm-i386 versions of headers get the
incompatible asm-um versions. The fix is to move the include/asm link
back to include/asm-i386 and to do UML builds someplace else.
This seems to be a similar situation with the resierfs problem above. Some
versions of NFS seems not to handle mmap correctly, which UML depends on.
The workaround is have /tmp be non-NFS directory.
If you build UML with gprof support and, early in the boot, it does this
kernel BUG at page_alloc.c:100!
you have a buggy gcc. You can work around the problem by removing
UM_FASTCALL from CFLAGS in arch/um/Makefile-i386. This will open up
another bug, but that one is fairly hard to reproduce.
This is a syslogd bug. There's a race between a parent process
installing a signal handler and its child sending the signal. See
this uml-devel post for the details.
It doesn't work on hosts running 2.4.7 (or thereabouts) or earlier. The fix
is to upgrade to something more recent and then read the next item.
If you see
File descriptor in bad state
when you
bring up the device inside UML, you have a header mismatch between the
original kernel and the upgraded one. Make /usr/src/linux point at
the new headers. This will only be a problem if you build uml_net
yourself.
If you can connect to the host, and the host can connect to UML, but you
can not connect to any other machines, then you may need to enable IP
Masquerading on the host. Usually this is only experienced when using
private IP addresses (192.168.x.x or 10.x.x.x) for host/UML
networking, rather than the public address space that your host
is connected to. UML does not enable IP Masquerading, so you will
need to create a static rule to enable it:
host%
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Replace eth0 with the interface that you use to talk to the
rest of the world.
Documentation on IP Masquerading, and SNAT, can be found at
www.netfilter.org .
If you can reach the local net, but not the outside Internet, then
that is usually a routing problem. The UML needs a default route:
UML#
route add default gw gateway IP
The gateway IP can be any machine on the local net that knows how to
reach the outside world. Usually, this is the host or the local
network's gateway.
Occasionally, we hear from someone who can reach some machines, but
not others on the same net, or who can reach some ports on other
machines, but not others. These are usually caused by strange
firewalling somewhere between the UML and the other box. You track
this down by running tcpdump on every interface the packets travel
over and see where they disappear. When you find a machine that
takes the packets in, but does not send them onward, that's the
culprit.
Thanks to Birgit Wahlich for telling me about this strange one. It
turns out that there's a limit of six environment variables on the
kernel command line. When that limit is reached or exceeded, argument
processing stops, which means that the 'root=' argument that UML
usually adds is not seen. So, the filesystem has no idea what the
root device is, so it panics.
The fix is to put less stuff on the command line. Glomming all your
setup variables into one is probably the best way to go.
On some older systems, /usr/include/asm/ptrace.h and
/usr/include/sys/ucontext.h define the same names. So, when they're
included together, the defines from one completely mess up the parsing
of the other, producing errors like:
/usr/include/sys/ucontext.h:47: parse error before
`10'
plus a pile of warnings.
This is a libc botch, which has since been fixed, and I don't see any
way around it besides upgrading.
On i386 kernels, there are two ways of running the loop that is used
to calculate the BogoMips rating, using the TSC if it's there or using
a one-instruction loop. The TSC produces twice the BogoMips as the
loop. UML uses the loop, since it has nothing resembling a TSC, and
will get almost exactly the same BogoMips as a host using the loop.
However, on a host with a TSC, its BogoMips will be double the loop
BogoMips, and therefore double the UML BogoMips.
If you're running an up to date kernel with an old release of
uml_utilities, the port-helper program will not work properly, so
xterms will exit straight after they appear. The solution is to
upgrade to the latest release of uml_utilities. Usually this problem
occurs when you have installed a packaged release of UML then
compiled your own development kernel without upgrading the
uml_utilities from the source distribution.
This problem is fixed by the skas-hold-own-ldt patch that went into
2.6.15-rc1.
The boot looks like this:
cannot set up thread-local storage: cannot set up LDT for thread-local storage
Kernel panic - not syncing: Attempted to kill init!
Your UML kernel doesn't support Native Posix Thread Library (NPTL) and
the binaries you're running are being dynamically linked to it. Try
running in SKAS3 mode first. You might be able to avoid the kernel
panic setting the
LD_ASSUME_KERNEL environment variable on the command line:
./linux init=/bin/sh LD_ASSUME_KERNEL=2.4.1
Many commands are very restrictive about what is preserved in the
environment when starting child processes, so relying on
LD_ASSUME_KERNEL to be globally set for all processes in the whole
system is generally not a good idea. It's very hard to
guarantee. Thus it's better to move the NPTL libraries away:
These appear to be fixed with the tls patches from Blaisorblade that
are currently in my
patchset . You can apply the entire
patchset, or you can move /lib/tls in the image away, as described
above.
If you're seeing truly strange behavior, such as hangs or panics that
happen in random places, or you try running the debugger to see what's
happening and it acts strangely, then it could be a problem in the
host kernel. If you're not running a stock Linus or -ac kernel, then
try that. An early version of the preemption patch and a 2.4.10 SuSE
kernel have caused very strange problems in UML.
Otherwise, let me know about it. Send a message to one of the UML
mailing lists - either the developer list - user-mode-linux-devel at
lists dot sourceforge dot net (subscription
info) or the user list - user-mode-linux-user at lists dot
sourceforge do net (subscription info),
whichever you prefer. Don't assume that everyone knows about it and
that a fix is imminent.
If you want to be super-helpful, read
Diagnosing Problems
and follow the instructions contained therein.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-14.html 0000644 0000000 0000000 00000015764 12742461304 017230 0 ustar
User Mode Linux HOWTO : Diagnosing ProblemsNextPreviousContents
If you get UML to crash, hang, or otherwise misbehave, you should
report this on one of the project mailing lists, either the
developer list - user-mode-linux-devel at lists dot sourceforge dot
net (subscription info) or the user list -
user-mode-linux-user at lists dot sourceforge dot net
(subscription info). When you do, it is
likely that I will want more information. So, it would be helpful to
read the stuff below, do whatever is applicable in your case, and
report the results to the list.
For any diagnosis, you're going to need to build a debugging kernel.
The binaries from this site aren't debuggable. If you haven't done
this before, read about
Compiling the kernel and modules and
Kernel debugging UML first.
The most common case is for a normal thread to panic. To debug this,
you will need to run it under the debugger (add 'debug' to the command
line). An xterm will start up with gdb running inside it. Continue
it when it stops in start_kernel and make it crash. Now ^C gdb and
'bt'. I'm going to want to see the resulting stack trace.
If the panic was a "Kernel mode fault", then there will be a segv
frame on the stack and I'm going to want some more information. The
stack might look something like this:
(UML gdb) backtrace
#0 0x1009bf76 in __sigprocmask (how=1, set=0x5f347940, oset=0x0)
at ../sysdeps/unix/sysv/linux/sigprocmask.c:49
#1 0x10091411 in change_sig (signal=10, on=1) at process.c:218
#2 0x10094785 in timer_handler (sig=26) at time_kern.c:32
#3 0x1009bf38 in __restore ()
at ../sysdeps/unix/sysv/linux/i386/sigaction.c:125
#4 0x1009534c in segv (address=8, ip=268849158, is_write=2, is_user=0)
at trap_kern.c:66
#5 0x10095c04 in segv_handler (sig=11) at trap_user.c:285
#6 0x1009bf38 in __restore ()
I'm going to want to see the symbol and line information for the value
of ip in the segv frame. In this case, you would do the following:
(UML gdb) i sym 268849158
and
(UML gdb) i line *268849158
The reason for this is the __restore frame right above the
segv_handler frame is hiding the frame that actually segfaulted. So,
I have to get that information from the faulting ip.
The less common and more painful case is when the tracing thread
panics. In this case, the kernel debugger will be useless because it
needs a healthy tracing thread in order to work. The first thing to
do is get a backtrace from the tracing thread. This is done by
figuring out what its pid is, firing up gdb, and attaching it to that
pid. You can figure out the tracing thread pid by looking at the
first line of the console output, which will look like this:
tracing thread pid = 15851
or by running ps on the host and finding the line that looks like
this:
However, there are cases where the misbehavior of another
thread caused the problem. The most common panic of this type is:
wait_for_stop failed to wait for <pid> to stop with <signal number>
In this case, you'll need to get a backtrace from the process
mentioned in the panic, which is complicated by the fact that the
kernel debugger is defunct and without some fancy footwork, another
gdb can't attach to it. So, this is how the fancy footwork goes:
In a shell:
host% kill -STOP pid
Run gdb on the tracing thread as described in case 2 and do:
(host gdb) call detach(pid)
If you get a segfault, do it again. It always works the second
time.
Detach from the tracing thread and attach to that other thread:
(host gdb) detach
(host gdb) attach pid
If gdb hangs when attaching to that process, go back to a shell and
do:
Hangs seem to be fairly rare, but they sometimes happen. When a hang
happens, we need a backtrace from the offending process. Run the
kernel debugger as described in case 1 and get a backtrace. If the
current process is not the idle thread, then send in the backtrace.
You can tell that it's the idle thread if the stack looks like this:
#0 0x100b1401 in __libc_nanosleep ()
#1 0x100a2885 in idle_sleep (secs=10) at time.c:122
#2 0x100a546f in do_idle () at process_kern.c:445
#3 0x100a5508 in cpu_idle () at process_kern.c:471
#4 0x100ec18f in start_kernel () at init/main.c:592
#5 0x100a3e10 in start_kernel_proc (unused=0x0) at um_arch.c:71
#6 0x100a383f in signal_tramp (arg=0x100a3dd8) at trap_user.c:50
If this is the case, then some other process is at fault, and went to
sleep when it shouldn't have. Run ps on the host and figure out which
process should not have gone to sleep and stayed asleep. Then attach
to it with gdb and get a backtrace as described in case 3.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-15.html 0000644 0000000 0000000 00000020000 12742461304 017204 0 ustar
User Mode Linux HOWTO : Thanks
Next
PreviousContents
A number of people have helped this project in various ways, and this page
gives recognition where recognition is due.
If you're listed here and you would prefer a real link on your name,
or no link at all, instead of the despammed email address pseudo-link,
let me know.
If you're not listed here and you think maybe you should be, please let
me know that as well. I try to get everyone, but sometimes my bookkeeping
lapses and I forget about contributions.
Lennert Buytenhek - Contributed various patches, a rewrite of the
network driver, the first implementation of the mconsole driver, and
did the bulk of the work needed to get SMP working again.
Yon Uriarte - Fixed the TUN/TAP network backend while I slept.
Adam Heath - Made a bunch of nice cleanups to the initialization code,
plus various other small patches.
Matt Zimmerman - Matt volunteered to be the UML Debian maintainer
and is doing a real nice job of it. He also noticed and fixed a
number of actually and potentially exploitable security holes in
uml_net. Plus the occasional patch. I like patches.
James McMechan - James seems to have taken over maintenance of the ubd
driver and is doing a nice job of it.
Chandan Kudige - wrote the umlgdb script which automates the reloading
of module symbols.
Steve Schmidtke - wrote the UML slirp transport and hostaudio drivers,
enabling UML processes to access audio devices on the host. He also
submitted patches for the slip transport and lots of other things.
SGI (and more specifically
Ralf Baechle ) gave me an account on
oss.sgi.com . The bandwidth
there made it possible to produce most of the filesystems available on
the project
download page.
Laurent Bonnaud took the old grotty
Debian filesystem that I've been distributing and updated it to 2.2.
It is now available by itself here.
Rik van Riel gave me some ftp space on ftp.nl.linux.org so I can
make releases even when Sourceforge is broken.
Rodrigo de Castro looked at my broken pte code and told me what
was wrong with it, letting me fix a long-standing (several weeks) and
serious set of bugs.
Chris Reahard built a specialized root filesystem for running a
DNS server jailed inside UML. It's available from the
download page in the
Jail Filesysems section.
Next
PreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-2.html 0000644 0000000 0000000 00000023526 12742461304 017140 0 ustar
User Mode Linux HOWTO : Compiling the kernel and modulesNextPreviousContents
Compiling the user mode kernel is just like compiling any other
kernel. Let's go through the steps, using 2.4.0-prerelease (current
as of this writing) as an example:
Download the latest UML patch from
the download page
In this example, the file is uml-patch-2.4.0-prerelease.bz2.
Run your favorite config; `make xconfig ARCH=um' is the most
convenient. `make config ARCH=um' and 'make menuconfig ARCH=um' will
work as well. The defaults will give you a useful kernel. If you
want to change something, go ahead, it probably won't hurt anything.
Note: If the host is configured with a 2G/2G address space split
rather than the usual 3G/1G split, then the packaged UML binaries will
not run. They will immediately segfault. See
UML on 2G/2G hosts for the scoop on
running UML on your system.
Finish with `make linux ARCH=um': the result is a file called `linux'
in the top directory of your source tree.
You may notice that the final binary is pretty large (many 10's of
megabytes for a debuggable UML). This is almost entirely symbol
information. The actual binary is comparable in size to a native
kernel. You can run that huge binary, and only the actual code and
data will be loaded into memory, so the symbols only consume disk
space unless you are running UML under gdb. You can strip UML:
host% strip linux
to see the true size of the UML kernel.
Make sure that you don't build this
kernel in /usr/src/linux. On some distributions, /usr/include/asm
is a link into this pool. The user-mode build changes the other end
of that link, and things that include <asm/anything.h> stop compiling.
The sources are also available from cvs. You can
browse the CVS pool
or access it anonymously via
If you get the CVS sources, you will have to check them out into an
empty directory. You will then have to copy each file into the corresponding
directory in the appropriate kernel pool.
If you don't have the latest kernel pool, you can get the corresponding
user-mode sources with
host% cvs co -r v_2_3_x linux
where 'x' is the version in your pool. Note that you will not get the bug
fixes and enhancements that have gone into subsequent releases.
If you build your own kernel, and want to boot it from one of the
filesystems distributed from this site, then, in nearly all cases,
devfs must be compiled into the kernel and mounted at boot time. The
exception is the tomsrtbt filesystem. For this,
devfs must either not be in the kernel at all, or "devfs=nomount" must
be on the kernel command line. Any disagreement between the kernel
and the filesystem being booted about whether devfs is being used will
result in the boot getting no further than single-user mode.
If you don't want to use devfs, you can remove the need for it from a
filesystem by copying /dev from someplace, making a bunch of
/dev/ubd devices:
UML#
for i in 0 1 2 3 4 5 6 7; do mknod ubd$i b 98 $[ $i * 16 ]; done
and changing /etc/fstab and /etc/inittab to refer to the non-devfs devices.
UML modules are built in the same way as the native kernel (with the
exception of the 'ARCH=um' that you always need for UML):
host% make modules ARCH=um
Any modules that you want to load into this kernel need to
be built in the user-mode pool. Modules from the native kernel won't
work. If you notice that the modules you get are much larger than
they are on the host, see the note above about the size of the final
UML binary.
You can install them by using ftp or something to copy them into the
virtual machine and dropping them into /lib/modules/`uname -r`.
You can also get the kernel build process to install them as
follows:
with the kernel not booted, mount the root filesystem in the top level
of the kernel pool:
host% mount root_fs mnt -o loop
run
host%
make modules_install INSTALL_MOD_PATH=`pwd`/mnt ARCH=um
unmount the filesystem
host% umount mnt
boot the kernel on it
If you can't mount the root filesystem on the host for some reason
(like it's a COW file), then an alternate approach is to mount the
UML kernel tree from the host into the UML with
hostfs and run the modules_install inside
UML:
With UML booted, mount the host kernel tree inside UML at the same
location as on the host:
UML# mount none -t hostfs path to UML pool -o
path to UML pool
Run make modules_install:
UML# cd path to UML pool ; make modules_install
The depmod at the end may complain about unresolved symbols because
there is an incorrect or missing System.map installed in the UML
filesystem. This appears to be harmless. insmod or modprobe should
work fine at this point.
When the system is booted, you can use insmod as usual to get the modules
into the kernel. A number of things have been loaded into UML as
modules, especially filesystems and network protocols and filters, so
most symbols which need to be exported probably already are. However,
if you do find symbols that need exporting, let
us know, and they'll be "taken care of".
If you try building an external module against a UML tree, you will
find that it doesn't compile because of missing includes. There are
less obvious problems with the CFLAGS that the module Makefile or
script provides which would make it not run even if it did build. To
get around this, you need to provide the same CFLAGS that the UML
kernel build uses.
A reasonably slick way of getting the UML CFLAGS is
cd uml-tree ; make script 'SCRIPT=@echo $(CFLAGS)' ARCH=um
If the module build process has something that looks like
$(CC) $(CFLAGS) file
then you can define CFLAGS in a script like this
CFLAGS=`cd uml-tree ; make script 'SCRIPT=@echo $(CFLAGS)' ARCH=um`
and like this in a Makefile
CFLAGS=$(shell cd uml-tree ; make script 'SCRIPT=@echo
$$(CFLAGS)' ARCH=um)
Many features of the UML kernel require a user-space helper program,
so a uml_utilities package is distributed separately from the kernel
patch which provides these helpers. Included within this is:
port-helper - Used by consoles which connect to xterms or ports
tunctl - Configuration tool to create and delete tap devices
uml_net - Setuid binary for automatic tap device configuration
uml_switch - User-space virtual switch required for daemon transport
The uml_utilities tree is compiled with:
host#
make && make install
Note that UML kernel patches may require a specific version of the
uml_utilities distribution. If you don't keep up with the mailing lists,
ensure that you have the latest release of uml_utilities if you are
experiencing problems with your UML kernel, particularly when dealing
with consoles or command-line switches to the helper programs
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-3.html 0000644 0000000 0000000 00000007725 12742461304 017144 0 ustar
User Mode Linux HOWTO : Running UML and logging inNextPreviousContents
It runs on 2.2.15 or later, and all 2.4 and 2.6 kernels.
Booting UML is straightforward. Simply run 'linux': it will try to
mount the file `root_fs' in the current directory. You do not need to
run it as root. If your root filesystem is not named `root_fs', then
you need to put a `ubd0=root_fs_whatever' switch on the linux command
line.
You will need a filesystem to boot UML from. There are a number
available for download from
here . There are also
several tools which can be used to generate UML-compatible filesystem
images from media.
The kernel will boot up and present you with a login prompt.
Note: If the host is configured with a 2G/2G address space split
rather than the usual 3G/1G split, then the packaged UML binaries will
not run. They will immediately segfault. See
UML on 2G/2G hosts for the scoop on
running UML on your system.
The prepackaged filesystems have a root account with
password 'root' and a user account with password 'user'. The
login banner will generally tell you how to log in. So, you log in
and you will find yourself inside a little virtual machine. Our
filesystems have a variety of commands and utilities installed (and it
is fairly easy to add more), so you will have a lot of tools with
which to poke around the system.
There are a couple of other ways to log in:
On a virtual console
Each virtual console that is configured (i.e. the
device exists in /dev and /etc/inittab runs a getty on it) will come up
in its own xterm. If you get tired of the xterms, read
Setting up serial lines and consoles to see how to
attach the consoles to something else, like host ptys.
Over the serial line
In the boot output, find a line that looks like:
serial line 0 assigned pty /dev/ptyp1
Attach your favorite terminal program to the corresponding
tty. I.e. for minicom, the command would be
host% minicom -o -p /dev/ttyp1
Over the net
If the network is running, then you can telnet to the virtual machine
and log in to it. See
Setting up the network to learn about setting up a virtual network.
When you're done using it, run halt, and the kernel
will bring itself down and the process will exit.
Most Linux machines are configured so that the kernel occupies the
upper 1G (0xc0000000 - 0xffffffff) of the 4G address space and
processes use the lower 3G (0x00000000 - 0xbfffffff). However, some
machine are configured with a 2G/2G split, with the kernel occupying
the upper 2G (0x80000000 - 0xffffffff) and processes using the lower
2G (0x00000000 - 0x7fffffff).
The prebuilt UML binaries on this site will not run on 2G/2G hosts
because UML occupies the upper .5G of the 3G process address space
(0xa0000000 - 0xbfffffff). Obviously, on 2G/2G hosts, this is right
in the middle of the kernel address space, so UML won't even load - it
will immediately segfault.
The fix for this is to rebuild UML from source after enabling
CONFIG_HOST_2G_2G (under 'General Setup'). This will cause UML to
load itself in the top .5G of that smaller process address space,
where it will run fine. See
Compiling the kernel and modules if you need help building UML from source.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-5.html 0000644 0000000 0000000 00000017330 12742461304 017137 0 ustar
User Mode Linux HOWTO : Setting up serial lines and consolesNextPreviousContents
Devices are specified with "con" or "ssl" (console or serial line,
respectively), optionally with a device number if you are talking
about a specific device.
Using just "con" or "ssl" describes all of the consoles or serial
lines. If you want to talk about console #3 or serial line #10, they
would be "con3" and "ssl10", respectively.
A specific device name will override a less general "con=" or "ssl=".
So, for example, you can assign a pty to each of the serial lines
except for the first two like this:
ssl=pty ssl0=tty:/dev/tty0 ssl1=tty:/dev/tty1
The specificity of the device name is all that matters; order on the
command line is irrelevant.
This will cause UML to allocate a free host pseudo-terminal for the
device. The terminal that it got will be announced in the boot log.
You access it by attaching a terminal program to the corresponding tty:
screen /dev/pts/n
screen /dev/ttyxx
minicom -o -p /dev/ttyxx - minicom seems not able to handle pts
devices
kermit - start it up, 'open' the device, then 'connect'
terminals - device=tty:tty device file
This will make UML attach the device to the specified tty (i.e
con1=tty:/dev/tty3
will attach UML's console 1 to the host's /dev/tty3). If the tty that
you specify is the slave end of a tty/pty pair, something
else must have already opened the corresponding pty in order for this
to work.
xterms - device=xterm
UML will run an xterm and the device will be attached to it.
Port - device=port:port number
This will attach the UML devices to the specified host port.
Attaching console 1 to the host's port 9000 would be done like this:
con1=port:9000
Attaching all the serial lines to that port would be done similarly:
ssl=port:9000
You access these devices by telnetting to that port. Each active
telnet session gets a different device. If there are more telnets to
a port than UML devices attached to it, then the extra telnet sessions
will block until an existing telnet detaches, or until another device
becomes active (i.e. by being activated in /etc/inittab).
This channel has the advantage that you can both attach multiple UML
devices to it and know how to access them without reading the UML boot
log. It is also unique in allowing access to a UML from remote
machines without requiring that the UML be networked. This could be
useful in allowing public access to UMLs because they would be
accessible from the net, but wouldn't need any kind of network
filtering or access control because they would have no network
access.
If you attach the main console to a portal, then the UML boot will
appear to hang. In reality, it's waiting for a telnet to connect, at
which point the boot will proceed.
If you set up a file descriptor on the UML command line, you can
attach a UML device to it. This is most commonly used to put the main
console back on stdin and stdout after assigning all the other
consoles to something else:
con0=fd:0,fd:1 con=pts
Nothing - device=null
This allows the device to be opened, in contrast to 'none', but reads
will block, and writes will succeed and the data will be thrown out.
None - device=none
This causes the device to disappear. If you are using devfs, the
device will not appear in /dev. If not, then attempts to open it will
return -ENODEV.
You can also specify different input and output channels for a device
by putting a comma between them:
ssl3=tty:/dev/tty2,xterm
will cause serial line 3 to accept input on the host's /dev/tty3 and
display output on an xterm. That's a silly example - the most common
use of this syntax is to reattach the main console to stdin and stdout
as shown above.
If you decide to move the main console away from stdin/stdout, the
initial boot output will appear in the terminal that you're running
UML in. However, once the console driver has been officially
initialized, then the boot output will start appearing wherever you
specified that console 0 should be. That device will receive all
subsequent output.
There are a number of interesting things you can do with this
capability.
First, this is how you get rid of those bleeding console xterms by
attaching them to host ptys:
con=pty con0=fd:0,fd:1
This will make a UML console take over an unused host virtual console,
so that when you switch to it, you will see the UML login prompt
rather than the host login
prompt:
con1=tty:/dev/tty6
You can attach two virtual machines together with what amounts to a
serial line as follows:
Run one UML with a serial line attached to a pty -
ssl1=pty
Look at the boot log to see what pty it got (this example will assume
that it got /dev/ptyp1).
Boot the other UML with a serial line attached to the corresponding
tty -
ssl1=tty:/dev/ttyp1
Log in, make sure that it has no getty on that serial line, attach
a terminal program like minicom to it, and you should see the login
prompt of the other virtual machine.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-6.html 0000644 0000000 0000000 00000073246 12742461304 017150 0 ustar
User Mode Linux HOWTO : Setting up the networkNextPreviousContents
This page describes how to set up the various transports and to
provide a UML instance with network access to the host, other machines
on the local net, and the rest of the net.
As of 2.4.5, UML networking has been completely redone to make it much
easier to set up, fix bugs, and add new features.
There is a new helper, uml_net, which does the host setup that
requires root privileges.
There are currently five transport types available for a UML virtual
machine to exchange packets with other hosts:
ethertap
TUN/TAP
Multicast
a switch daemon
slip
slirp
pcap
The TUN/TAP, ethertap, slip, and slirp transports allow a UML instance to
exchange packets with the host. They may be directed to the host or
the host may just act as a router to provide access to other physical
or virtual machines.
The pcap transport is a synthetic read-only interface, using the libpcap
binary to collect packets from interfaces on the host and filter them.
This is useful for building preconfigured traffic monitors or sniffers.
The daemon and multicast transports provide a completely virtual
network to other virtual machines. This network is completely
disconnected from the physical network unless one of the virtual
machines on it is acting as a gateway.
With so many host transports, which one should you use? Here's when
you should use each one:
ethertap - if you want access to the host networking and it is running
2.2
TUN/TAP - if you want access to the host networking and it is running
2.4. Also, the TUN/TAP transport is able to use a preconfigured
device, allowing it to avoid using the setuid uml_net helper, which
is a security advantage.
Multicast - if you want a purely virtual network and you don't want to
set up anything but the UML
a switch daemon - if you want a purely virtual network and you don't
mind running the daemon in order to get somewhat better performance
slip - there is no particular reason to run the slip backend unless
ethertap and TUN/TAP are just not available for some reason
slirp - if you don't have root access on the host to setup networking,
or if you don't want to allocate an IP to your UML
pcap - not much use for actual network connectivity, but great for
monitoring traffic on the host
Ethertap is available on 2.4 and works fine. TUN/TAP is preferred to
it because it has better performance and ethertap is officially
considered obsolete in 2.4. Also, the root helper only needs to run
occasionally for TUN/TAP, rather than handling every packet, as it
does with ethertap. This is a slight security advantage since it
provides fewer opportunities for a nasty UML user to somehow exploit
the helper's root privileges.
First, you must have the virtual network enabled in your UML. If are
running a prebuilt kernel from this site, everything is already enabled.
If you build the kernel yourself, under the "Network device support" menu,
enable "Network device support", and then the three transports.
The next step is to provide a network device to the virtual machine.
This is done by describing it on the kernel command line.
The general format is
eth <n> = <transport> , <transport args>
For example, a virtual ethernet device may be attached to a host ethertap
device as follows:
eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
This sets up eth0 inside the virtual machine to attach itself to the
host /dev/tap0, assigns it an ethernet address, and assigns the host
tap0 interface an IP address.
Note that the IP address you assign to the host end of the tap device
must be different than the IP you assign to the eth device inside
UML. If you are short on IPs and don't want to comsume two per
UML, then you can reuse the host's eth IP address for the host ends of
the tap devices. Internally, the UMLs must still get unique IPs for
their eth devices. You can also give the UMLs non-routable IPs
(192.168.x.x or 10.x.x.x) and have the host masquerade them. This
will let outgoing connections work, but incoming connections won't
without more work, such as port forwarding from the host.
Also note that when you configure the host side of an interface, it is only
acting as a gateway. It will respond to pings sent to it locally, but
is not useful to do that since it's a host interface. You are not
talking to the UML when you ping that interface and get a response.
You can also add devices to a UML and remove them at runtime. See
the
The Management Console page for details.
The sections below describe this in more detail.
Once you've decided how you're going to set up the devices, you boot
UML, log in, configure the UML side of the devices, and set up routes
to the outside world. At that point, you will be able to talk to any
other machines, physical or virtual, on the net.
If ifconfig inside UML fails and the network refuses to come up, run
'dmesg' to see what ended up in the kernel log. That will usually
tell you what went wrong.
You will likely need the setuid helper, or the switch daemon, or
both. They are both installed with the RPM and deb, so if you've installed
either, you can skip the rest of this section.
If not, then you need to check them out of CVS,
build them, and install them. The helper is uml_net, in CVS
/tools/uml_net, and the daemon is uml_switch, in CVS
/tools/uml_router. They are both built with a plain 'make'. Both
need to be installed in a directory that's in your path - /usr/bin is
recommend. On top of that, uml_net needs to be setuid root.
Below, you will see that the TUN/TAP, ethertap, and daemon interfaces
allow you to specify hardware addresses for the virtual ethernet
devices. This is generally not necessary. If you don't have a
specific reason to do it, you probably shouldn't. If one is not
specified on the command line, the driver will assign one based on the
device IP address. It will provide the address fe:fd:nn:nn:nn:nn
where nn.nn.nn.nn is the device IP address. This is nearly always
sufficient to guarantee a unique hardware address for the device. A
couple of exceptions are:
Another set of virtual ethernet devices are on the same network and
they are assigned hardware addresses using a different scheme which
may conflict with the UML IP address-based scheme
You aren't going to use the device for IP networking, so you don't
assign the device an IP address
If you let the driver provide the hardware address, you should make sure
that the device IP address is known before the interface is brought
up. So, inside UML, this will guarantee that:
UML#
ifconfig eth0 192.168.0.250 up
If you decide to assign the hardware address yourself, make sure that
the first byte of the address is even. Addresses with an odd first byte
are broadcast addresses, which you don't want assigned to a device.
Once the network devices have been described on the command line, you
should boot UML and log in.
The first thing to do is bring the interface up:
UML# ifconfig ethn ip-address up
You should be able to ping the host at this point.
To reach the rest of the world, you should set a default route to the
host:
UML# route add default gw host ip
Again, with host ip of 192.168.0.4:
UML# route add default gw 192.168.0.4
This page used to recommend setting a network route to your local
net. This is wrong, because it will cause UML to try to figure out
hardware addresses of the local machines by arping on the interface to
the host. Since that interface is basically a single strand of
ethernet with two nodes on it (UML and the host) and arp requests
don't cross networks, they will fail to elicit any responses. So,
what you want is for UML to just blindly throw all packets at the host
and let it figure out what to do with them, which is what leaving out
the network route and adding the default route does.
Note: If you can't communicate with other hosts on your physical
ethernet, it's probably because of a network route that's
automatically set up. If you run 'route -n' and see a route that
looks like this:
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
with a mask that's not 255.255.255.255, then replace it with a route
to your host:
UML#
route del -net 192.168.0.0 dev eth0 netmask 255.255.255.0
UML#
route add -host 192.168.0.4 dev eth0
This, plus the default route to the host, will allow UML to exchange
packets with any machine on your ethernet.
The simplest way to set up a virtual network between multiple UMLs is
to use the mcast transport. This was written by Harald Welte and is
present in UML version 2.4.5-5um and later. Your system must have
multicast enabled in the kernel and there must be a multicast-capable
network device on the host. Normally, this is eth0, but if there is
no ethernet card on the host, then you will likely get strange error
messages when you bring the device up inside UML.
To use it, run two UMLs with
eth0=mcast
on their command lines. Log in, configure the ethernet device in each
machine with different IP addresses:
UML1# ifconfig eth0 192.168.0.254
UML2# ifconfig eth0 192.168.0.253
and they should be able to talk to each other.
The full set of command line options for this transport are
TUN/TAP is the preferred mechanism on 2.4 to exchange packets with the
host. The TUN/TAP backend has been in UML since 2.4.9-3um.
The easiest way to get up and running is to let the setuid uml_net
helper do the host setup for you. This involves insmod-ing the tun.o
module if necessary, configuring the device, and setting up IP
forwarding, routing, and proxy arp. If you are new to UML networking,
do this first. If you're concerned about the security implications of
the setuid helper, use it to get up and running, then read the next
section to see how to have UML use a preconfigured tap device, which
avoids the use of uml_net.
If you specify an IP address for the host side of the device, the
uml_net helper will do all necessary setup on the host - the only
requirement is that TUN/TAP be available, either built in to the host
kernel or as the tun.o module.
The format of the command line switch to attach a device to a TUN/TAP device
is
eth <n> =tuntap,,, <host IP address>
For example, this argument will attach the UML's eth0 to
the next available tap device, assign the IP address
192.168.0.254 to the host side of the tap device, and
assign an ethernet address to it based on the IP address assigned to
it by ifconfig inside UML.
eth0=tuntap,,,192.168.0.254
If you using the uml_net helper to set up the host side of the
networking, as in this example, note that changing the UML IP address
will cause uml_net to change the host routing and arping to match.
This is one reason you should not be using uml_net if there is any
possibility that the user inside the UML may be unfriendly. This
feature is convenient, but can be used to make the UML pretend to be
something like your name server or mail server, and the host will
steal packets intended for those servers and forward them to the UML.
See the next section for setting up networking in a secure manner.
There are a couple potential problems with running the TUN/TAP
transport on a 2.4 host kernel
TUN/TAP seems not to work on 2.4.3 and earlier. Upgrade the host
kernel or use the ethertap transport.
With an upgraded kernel, TUN/TAP may fail with
File descriptor in bad state
This is due to a header mismatch between the upgraded kernel and the
kernel that was originally installed on the machine. The fix is to
make sure that /usr/src/linux points to the headers for the running kernel.
If you prefer not to have UML use uml_net (which is somewhat
insecure), with UML 2.4.17-11, you can set up a TUN/TAP device
beforehand. The setup needs to be done as root, but once that's done,
there is no need for root assistance. Setting up the device is done
as follows:
Create the device with tunctl (available from the UML utilities
tarball)
host# tunctl -u uid
where uid is the user id or username that UML will be run as.
This will tell you what device was created.
Configure the device IP (change IP addresses and device name to suit)
host# ifconfig tap0 192.168.0.254 up
Set up routing and arping if desired - this is my recipe, there are
other ways of doing the same thing
Note that this must be done every time the host boots - this
configuration is not stored across host reboots. So, it's probably a
good idea to stick it in an rc file. An even better idea would be a
little utility which reads the information from a config file and sets
up devices at boot time.
Rather than using up two IPs and ARPing for one of them, you can also
provide direct access to your LAN by the UML by using a bridge.
host#
brctl addbr br0
host#
ifconfig eth0 0.0.0.0 promisc up
host#
ifconfig tap0 0.0.0.0 promisc up
host#
ifconfig br0 192.168.0.1 netmask 255.255.255.0 up
host#
brctl stp br0 off
host#
brctl setfd br0 1
host#
brctl sethello br0 1
host#
brctl addif br0 eth0
host#
brctl addif br0 tap0
Note that 'br0' should be setup using ifconfig with the existing IP
address of eth0, as eth0 no longer has its own IP.
Also, the /dev/net/tun device must be writable by the user running UML
in order for the UML to use the device that's been configured for it.
The simplest thing to do is
host# chmod 666 /dev/net/tun
Making it world-writeable looks bad, but it seems not to be
exploitable as a security hole. However, it does allow anyone to
create useless tap devices (useless because they can't configure
them), which is a DOS attack. A somewhat more secure alternative
would to be to create a group containing all the users who have
preconfigured tap devices and chgrp /dev/net/tun to that group with
mode 664 or 660.
Once the device is set up, run UML with
eth0=tuntap,devicename
i.e.
eth0=tuntap,tap0
on the command line (or do it with
the mconsole config command).
Bring the eth device up in UML and you're in business.
If you don't want that tap device any more, you can make it
non-persistent with
host# tunctl -d tap device
Finally, tunctl has a -b (for brief mode) switch which causes it to
output only the name of the tap device it created. This makes it
suitable for capture by a script:
Ethertap is the general mechanism on 2.2 for userspace processes to
exchange packets with the kernel.
To use this transport, you need to describe the virtual network
device on the UML command line. The general format for this is
eth <n> =ethertap, <device> , <ethernet address> , <host IP address>
So, the previous example
eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
attaches the UML eth0 device to the host
/dev/tap0, assigns it the
ethernet address fe:fd:0:0:0:1, and assigns the IP address
192.168.0.254 to the host side of the tap device.
The tap device is mandatory, but the others are optional. If the
ethernet address is omitted, one will be assigned to it.
The presence of the tap IP address will cause the helper to run and do
whatever host setup is needed to allow the virtual machine to
communicate with the outside world. If you're not sure you know what
you're doing, this is the way to go.
If it is absent, then you must configure the tap device and whatever
arping and routing you will need on the host. However, even in this
case, the uml_net helper still needs to be in your path and it must be
setuid root if you're not running UML as root. This is because the
tap device doesn't support SIGIO, which UML needs in order to use
something as a source of input. So, the helper is used as a
convenient asynchronous IO thread.
If you're using the uml_net helper, you can ignore the following host
setup - uml_net will do it for you. You just need to make sure you
have ethertap available, either built in to the host kernel or
available as a module.
If you want to set things up yourself, you need to make
sure that the appropriate /dev entry exists. If it doesn't, become
root and create it as follows (the $[ ... ] is bash syntax for adding 16 to
the minor number) :
mknod /dev/tap <minor> c 36 $[ <minor> + 16 ]
For example, this is how to create /dev/tap0:
mknod /dev/tap0 c 36 $[ 0 + 16 ]
You also need to make sure that the host kernel has ethertap support.
If ethertap is enabled as a module, you apparently need to insmod
ethertap once for each ethertap device you want to enable. So,
host#
insmod ethertap
will give you the tap0 interface. To get the tap1 interface, you need
to run
Note: This is the daemon formerly known as uml_router, but
which was renamed so the network weenies of the world would stop
growling at me.
The switch daemon, uml_switch, provides a mechanism for creating a
totally virtual network. By default, it provides no connection to the
host network (but see -tap, below).
The first thing you need to do is run the daemon. Running it with no
arguments will make it listen on a default unix domain socket.
If you want it to listen on a different socket, use
-unix socket
If you want it to act as a hub rather than a switch, use
-hub
If you're planning on putting it in hub mode so you can sniff UML
traffic from a tap device on the host, it appears that you need to
assign the tap an IP address before you'll see any packets on it.
If you want the switch to be connected to host networking (allowing the
umls to get access to the outside world through the host), use
-tap tap0
Note that the tap device must be preconfigured (see "TUN/TAP with a
preconfigured tap device", above). If you're using a different tap
device than tap0, specify that instead of tap0.
The reason it doesn't background by default is that it listens to
stdin for EOF. When it sees that, it exits.
The general format of the kernel command line switch is
ethn=daemon,ethernet address,socket type,socket
You can leave off everything except the 'daemon'. You only need to
specify the ethernet address if the one that will be assigned to it
isn't acceptable for some reason. The rest of the arguments describe
how to communicate with the daemon. You should only specify them if
you told the daemon to use different sockets than the default. So, if
you ran the daemon with no arguments, running the UML on the same
machine with
eth0=daemon
will cause the eth0 driver to attach itself to the daemon correctly.
The socket argument is the filename of a Unix domain socket which is
used for communications between uml_switch and the UMLs on its network.
If you do specify a different socket from the default, which you will
need to do if you want multiple, separate uml_switch networks on the
host, you need to make sure that you name the same path for the socket
on both the uml_switch and UML command lines.
Currently the only supported value for the socket type is "unix".
Slip is another, less general, mechanism for a process to communicate
with the host networking. In contrast to the ethertap interface,
which exchanges ethernet frames with the host and can be used to
transport any higher-level protocol, it can only be used to transport
IP.
The general format of the command line switch is
ethn=slip,slip IP
The slip IP argument is the IP address that will be assigned to the
host end of the slip device. If it is specified, the helper will run
and will set up the host so that the virtual machine can reach it and
the rest of the network.
There are some oddities with this interface that you should be aware
of. You should only specify one slip device on a given virtual
machine, and its name inside UML will be 'umn', not 'eth0' or whatever
you specified on the command line. These problems will be fixed at some point.
slirp uses an external program, usually /usr/bin/slirp, to provide IP
only networking connectivity through the host. This is similar to IP
masquerading with a firewall, although the translation is performed in
user-space, rather than by the kernel. As slirp does not set up any
interfaces on the host, or changes routing, slirp does not require
root access or setuid binaries on the host.
The general format of the command line switch for slirp is:
ethn=slirp,ethernet address,slirp path
The ethernet address is optional, as UML will set up the interface
with an ethernet address based upon the initial IP address of the
interface. The slirp path is generally /usr/bin/slirp, although it
will depend on distribution.
The slirp program can have a number of options passed to the command line
and we can't add them to the UML command line, as they will be parsed
incorrectly. Instead, a wrapper shell script can be written or the options
inserted into the /.slirprc file. More information on all of the slirp
options can be found in its man pages.
The eth0 interface on UML should be set up with the IP 10.2.0.15,
although you can use anything as long as it is not used by a network you
will be connecting to. The default route on UML should be set to use
'eth0' without a gateway IP:
UML#
route add default dev eth0
slirp provides a number of useful IP addresses which can be used by
UML, such as 10.0.2.3 which is an alias for the DNS server specified in
/etc/resolv.conf on the host or the IP given in the 'dns' option for slirp.
Even with a baudrate setting higher than 115200, the slirp connection
is limited to 115200. If you need it to go faster, the slirp binary
needs to be compiled with FULL_BOLT defined in config.h.
The interface is whatever network device on the host you want to sniff.
The expression is a pcap filter expression, which is also what tcpdump
uses, so if you know how to specify tcpdump filters, you will use the
same expressions here. The options are up to two of 'promisc',
'nopromisc', 'optimize', 'nooptimize'. 'promisc' and 'nopromisc'
control whether pcap puts the host interface into promiscuous
mode. 'optimize' and 'nooptimize' control whether the pcap expression
optimizer is used.
Example:
eth0=pcap,eth0,tcp
eth1=pcap,eth0,!tcp
will cause the UML eth0 to emit all tcp packets on the host eth0 and the
UML eth1 to emit all non-tcp packets on the host eth0.
If you don't specify an address for the host side of the ethertap or
slip device, UML won't do any setup on the host. So this is what is
needed to get things working (the examples use a host-side IP of
192.168.0.251 and a UML-side IP of 192.168.0.250 - adjust to suit your
own network):
The device needs to be configured with its IP address. Tap devices
are also configured with an mtu of 1484. Slip devices are configured
with a point-to-point address pointing at the UML ip address.
host# ifconfig tap0 arp mtu 1484 192.168.0.251 up
host#
ifconfig sl0 192.168.0.251 pointopoint 192.168.0.250 up
If a tap device is being set up, a route is set to the UML IP.
Don't attempt to share filesystems simply by booting two UMLs from the
same file. That's the same thing as booting two physical machines
from a shared disk. It will result in filesystem corruption.
The way to share a filesystem between two virtual machines is to use
the copy-on-write (COW) layering capability of the ubd block driver. As of
2.4.6-2um, the driver supports layering a read-write private device
over a read-only shared device. A machine's writes are stored in the
private device, while reads come from either device - the private one
if the requested block is valid in it, the shared one if not. Using
this scheme, the majority of data which is unchanged is shared between
an arbitrary number of virtual machines, each of which has a much
smaller file containing the changes that it has made. With a large
number of UMLs booting from a large root filesystem, this leads to a
huge disk space saving. It will also help performance, since the host
will be able to cache the shared data using a much smaller amount of
memory, so UML disk requests will be served from the host's memory
rather than its disks.
To add a copy-on-write layer to an existing block device file, simply
add the name of the COW file to the appropriate ubd switch:
ubd0=root_fs_cow,root_fs_debian_22
where 'root_fs_cow' is the private COW file and 'root_fs_debian_22' is
the existing shared filesystem. The COW file need not exist. If it
doesn't, the driver will create and initialize it. Once the COW file
has been initialized, it can be used on its own on the command line:
ubd0=root_fs_cow
The name of the backing file is stored in the COW file header, so it
would be redundant to continue specifying it on the command line.
When checking the size of the COW file in order to see the gobs of
space that you're saving, make sure you use 'ls -ls' to see the actual
disk consumption rather than the length of the file. The COW file is
sparse, so the length will be very different from the disk usage.
Here is a 'ls -l' of a COW file and backing file from one boot and
shutdown:
host% ls -l cow.debian debian2.2
-rw-r--r-- 1 jdike jdike 492504064 Aug 6 21:16 cow.debian
-rwxrw-rw- 1 jdike jdike 537919488 Aug 6 20:42 debian2.2
Doesn't look like much saved space, does it? Well, here's 'ls -ls':
host% ls -ls cow.debian debian2.2
880 -rw-r--r-- 1 jdike jdike 492504064 Aug 6 21:16 cow.debian
525832 -rwxrw-rw- 1 jdike jdike 537919488 Aug 6 20:42 debian2.2
Now, you can see that the COW file has less than a meg of disk, rather
than 492 meg.
Once a filesystem is being used as a readonly backing file for a COW
file, do not boot directly from it or modify it in any way. Doing so
will invalidate any COW files that are using it. The mtime and size
of the backing file are stored in the COW file header at its creation,
and they must continue to match. If they don't, the driver will
refuse to use the COW file.
If you attempt to evade this restriction by changing either the
backing file or the COW header by hand, you will get a corrupted
filesystem.
Among other things, this means that upgrading the distribution in a
backing file and expecting that all of the COW files using it will see
the upgrade will not work.
Because UML stores the backing file name and its mtime in the COW
header, if you move the backing file, that information becomes
invalid. So, the procedure for moving a backing file is
Move it in a way that preserves timestamps. Usually, this is a "-p"
switch. "cp -a" works because "-a" implies "-p".
Update the COW header by booting UML on it, specifying both the COW
file and the new location of the backing file
host% ubd0=COW file,new backing file
location
UML will notice the mismatch between the command line and COW header,
check the size and mtime of the new backing file path, and update the
COW header to reflect it if it checks out.
If you forget to preserve the timestamps when you move the backing
file, you can fix the mtime by hand as follows
host%
mtime=whatever UML says mtime should be ; \
touch --date="`date -d 1970-01-01\ UTC\ $mtime\ seconds`" backing file
Note that if you do this on a backing file that has truly been
changed, and not just moved, then you will get file corruption and you
will lose the filesystem.
Depending on how you use UML and COW devices, it may be advisable to
merge the changes in the COW file into the backing file every once in
a while.
The utility that does this is uml_moo. Its usage is
host% uml_moo COW file new backing file
There's no need to specify the backing file since that information is
already in the COW file header. If you're paranoid, boot the new
merged file, and if you're happy with it, move it over the old backing
file.
uml_moo creates a new backing file by default as a safety measure. It
also has a destructive merge option which will merge the COW file
directly into its current backing file. This is really only usable
when the backing file only has one COW file associated with it. If
there are multiple COWs associated with a backing file, a -d merge of
one of them will invalidate all of the others. However, it is
convenient if you're short of disk space, and it should also be
noticably faster than a non-destructive merge. This usage is
host% uml_moo -d COW file
uml_moo is installed with the UML deb and RPM. If you didn't install
UML from one of those packages, you can also get it from the
UML utilities tar file in tools/moo.
The normal way to create a COW file is to specify a non-existant COW
file on the UML command line, and let UML create it for you. However,
sometimes you want a new COW file, and you don't want to boot UML in
order to get it. This can be done with uml_mkcow, which is a little
standalone utility by Steve Schnepp.
The standard usage is
host% uml_mkcow new COW file existing
backing file
If you want to destroy an existing COW file, then there is a -f switch
to force the overwriting of the old COW file
You may want to create and mount new UML filesystems, either because
your root filesystem isn't large enough or because you want to use a
filesystem other than ext2.
This was written on the occasion of reiserfs being included in the
2.4.1 kernel pool, and therefore the 2.4.1 UML, so the examples will
talk about reiserfs. This information is generic, and the examples
should be easy to translate to the filesystem of your choice.
dd is your friend. All you need to do is tell dd to create an empty
file of the appropriate size. I usually make it sparse to save time
and to avoid allocating disk space until it's actually used. For
example, the following command will create a sparse 100 meg file full
of zeroes.
Make sure that the filesystem is available, either by being built into
the kernel, or available as a module, then boot up UML and log in. If
the root filesystem doesn't have the filesystem utilities (mkfs, fsck,
etc), then get them into UML by way of the net or hostfs.
Make the new filesystem on the device assigned to the new file:
host# mkreiserfs /dev/ubd/4
<----------- MKREISERFSv2 ----------->
ReiserFS version 3.6.25
Block size 4096 bytes
Block count 25856
Used blocks 8212
Journal - 8192 blocks (18-8209), journal header is in block 8210
Bitmaps: 17
Root block 8211
Hash function "r5"
ATTENTION: ALL DATA WILL BE LOST ON '/dev/ubd/4'! (y/n)y
journal size 8192 (from 18)
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..done.
Now, mount it:
UML#
mount /dev/ubd/4 /mnt
and you're in business.
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO-9.html 0000644 0000000 0000000 00000012453 12742461304 017144 0 ustar
User Mode Linux HOWTO : Host file accessNextPreviousContents
If you want to access files on the host machine from inside UML, you
can treat it as a separate machine and either nfs mount directories
from the host or copy files into the virtual machine with scp or rcp.
However, since UML is running on the the host, it can access those
files just like any other process and make them available inside the
virtual machine without needing to use the network.
This is now possible with the hostfs virtual filesystem. With it, you
can mount a host directory into the UML filesystem and access the
files contained in it just as you would on the host.
Note that hostfs is currently not available on 2.5. The reason is
that there was an fs.h rework early in 2.5 which required filesystem
changes, and I haven't got around to updating hostfs to those changes.
To begin with, make sure that hostfs is available inside the virtual
machine with
UML# cat /proc/filesystems
hostfs should be listed. If it's not, either rebuild the kernel with
hostfs configured into it or make sure that hostfs is built as a
module and available inside the virtual machine, and insmod it.
Now all you need to do is run mount:
UML# mount none /mnt/host -t hostfs
will mount the host's / on the virtual machine's /mnt/host.
If you don't want to mount the host root directory, then you can
specify a subdirectory to mount with the -o switch to mount:
UML# mount none /mnt/home -t hostfs -o /home
will mount the hosts's /home on the virtual machine's /mnt/home.
There is a hostfs option available on the UML command line which can
be used confine all hostfs mounts to a host directory hierarchy or to
prevent a hostfs user from destroying data on the host. The format is
hostfs=directory,options
The only option available at present is 'append', which forces all
files to be opened in append mode and disallows any deletion of files.
To specify append mode without confining hostfs to a host directory,
just leave out the directory name so that the argument begins with a
comma:
If you need to build hostfs because it's not in your kernel, you have
two choices:
Compiling hostfs into the kernel:
Reconfigure the kernel and set the 'Host filesystem' option under
'Processor features' to 'Y'. Recompile the kernel and reboot it.
Compiling hostfs as a module:
Reconfigure the kernel and set the 'Host filesystem' option under
'Processor features' to 'M'. Rebuild the kernel modules. hostfs will
be in arch/um/fs/hostfs/hostfs.o. Install that in /lib/modules/`uname -r`/fs
in the virtual machine, boot it up, and
UML# insmod hostfs
NextPreviousContents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO.html 0000644 0000000 0000000 00000026722 12742461304 017002 0 ustar
User Mode Linux HOWTO Next
Previous
Contents
User Mode Linux HOWTO
User Mode Linux Core Team
Mon Jan 30 16:52:10 EST 2006
This document describes the use and abuse of Jeff Dike's User Mode Linux: a port of the Linux kernel as a normal Intel Linux process.
Next
Previous
Contents
user-mode-linux-doc-20060501/UserModeLinux-HOWTO.txt 0000644 0000000 0000000 00000415315 12742461304 016655 0 ustar User Mode Linux HOWTO
User Mode Linux Core Team
Mon Jan 30 16:52:10 EST 2006
This document describes the use and abuse of Jeff Dike's User Mode
Linux: a port of the Linux kernel as a normal Intel Linux process.
______________________________________________________________________
Table of Contents
1. Introduction
1.1 What is User Mode Linux?
1.2 How is User Mode Linux Different?
1.3 How does UML Work?
1.4 Why Would I Want UML?
2. Compiling the kernel and modules
2.1 Compiling the kernel
2.2 Compiling and installing kernel modules
2.3 Compiling and installing uml_utilities
3. Running UML and logging in
3.1 Running UML
3.2 Logging in
3.3 Examples
4. UML on 2G/2G hosts
4.1 Introduction
4.2 The problem
4.3 The solution
5. Setting up serial lines and consoles
5.1 Specifying the device
5.2 Specifying the channel
5.3 Examples
6. Setting up the network
6.1 General setup
6.2 Userspace daemons
6.3 Specifying ethernet addresses
6.4 UML interface setup
6.5 Multicast
6.6 TUN/TAP with the uml_net helper
6.7 TUN/TAP with a preconfigured tap device
6.8 Ethertap
6.9 The switch daemon
6.10 Slip
6.11 Slirp
6.12 pcap
6.13 Setting up the host yourself
7. Sharing Filesystems between Virtual Machines
7.1 A warning
7.2 Using layered block devices
7.3 Note!
7.4 Another warning
7.5 Moving a backing file
7.6 uml_moo : Merging a COW file with its backing file
7.7 uml_mkcow : Create a new COW file
8. Creating filesystems
8.1 Create the filesystem file
8.2 Assign the file to a UML device
8.3 Creating and mounting the filesystem
9. Host file access
9.1 Using hostfs
9.2 hostfs command line options
9.3 hostfs as the root filesystem
9.4 Building hostfs
10. The Management Console
10.1 version
10.2 halt and reboot
10.3 config
10.4 remove
10.5 sysrq
10.6 help
10.7 cad
10.8 stop
10.9 go
10.10 log
10.11 proc
10.12 Making online backups
10.13 Event notification
11. Kernel debugging
11.1 Starting the kernel under gdb
11.2 Examining sleeping processes
11.3 Running ddd on UML
11.4 Debugging modules
11.5 Attaching gdb to the kernel
11.6 Using alternate debuggers
12. Kernel debugging examples
12.1 The case of the hung fsck
12.2 Episode 2: The case of the hung fsck
13. What to do when UML doesn't work
13.1 Strange compilation errors when you build from source
13.2 UML hangs on boot after mounting devfs
13.3 A variety of panics and hangs with /tmp on a reiserfs filesystem
13.4 The compile fails with errors about conflicting types for 'open', 'dup', and 'waitpid'
13.5 UML doesn't work when /tmp is an NFS filesystem
13.6 UML hangs on boot when compiled with gprof support
13.7 syslogd dies with a SIGTERM on startup
13.8 TUN/TAP networking doesn't work on a 2.4 host
13.9 You can network to the host but not to other machines on the net
13.10 I have no root and I want to scream
13.11 UML build conflict between ptrace.h and ucontext.h
13.12 The UML BogoMips is exactly half the host's BogoMips
13.13 When you run UML, it immediately segfaults
13.14 xterms appear, then immediately disappear
13.15 cannot set up thread-local storage
13.16 Process segfaults with a modern (NPTL-using) filesystem
13.17 Any other panic, hang, or strange behavior
14. Diagnosing Problems
14.1 Case 1 : Normal kernel panics
14.2 Case 2 : Tracing thread panics
14.3 Case 3 : Tracing thread panics caused by other threads
14.4 Case 4 : Hangs
15. Thanks
15.1 Code and Documentation
15.2 Flushing out bugs
15.3 Buglets and clean-ups
15.4 Case Studies
15.5 Other contributions
______________________________________________________________________
[1m1. Introduction[0m
Welcome to User Mode Linux. It's going to be fun.
[1m1.1. What is User Mode Linux?[0m
User Mode Linux lets you run Linux inside itself! With that comes the
power to do all sorts of new things. It virtualises (or simulates, as
some people call it) Linux so that you can run an entire Linux where
once you would have only run a program.
You might have heard of functionality like this before. There are
quite a few projects whose goal is to nest operating systems in one
way or another: Linux on Linux, Windows on Linux, Linux on Windows,
Linux/s390 on Linux/anythingelse, and so on. Or even just x86 on
anything, where the 'x86' program can boot operating systems including
Linux.
Where x86 is involved there is the greatest concentration of efforts.
At the end of this HOWTO you'll find a list of alternative projects.
If all you want to do is run a copy of x86 Linux on another copy of
x86 Linux as fast as possible with little control then quite possibly
one of these other projects will so better than UML.
[1m1.2. How is User Mode Linux Different?[0m
User Mode Linux (UML) is rather different from every other Linux
virtualisation project, either free or commercial. UML strives to
present itself as an ordinary program as much as possible. Here are
some of the outcomes of that philosophy:
1. Good speed with few compromises. UML compiles to native machine
code that runs just like any other compiled application on the
host. This makes it very much faster than portable virtualisation
schemes that implement an entire hardware architecture in software.
On the other hand, UML does not suffer from the extreme hardware
specificity of virtualisation systems that rely on particular CPU
features. UML runs applications inside itself with normally at
worst a 20% slowdown compared to the host system, which modern
hardware and clever system design can render negligable in real
terms.
2. Futureproof. Every time Linux gets improved so it can do something
new and clever that benefits the programs it runs, UML
automatically gets that facility. Software suspend, fine-grained
security control such as SE Linux, new filesystem features, support
for bigger/faster hardware... the same is not true with those
virtualisation systems that require major changes on the host
computer.
3. Flexible code. Normally an OS kernel is just that... a kernel. It
talks to hardware or maybe some virtualised hardware. But UML can
be viewed in many other ways. It would be possible to turn it into
a shared library, for example, so that other programs could link to
it to take advantage of things that Linux does very well. It can be
started as a subshell of an existing application. It can use
stin/stdout like any other program.
4. Portable. Really portable. UML has only just started to be
exploited for its portability, but there is promising evidence that
ports to x86 Windows, PowerPC Linux, x86 BSD and other systems are
very practical.
5. Mature. UML has been in development since 1999. One indication of
its robustness is that UML can be compiled to run within UML,
making it 'self-hosting'. Production systems are running on UML.
6. Free Software. UML is under the GPL (as it must be, being part of
the Linux kernel.)
[1m1.3. How does UML Work?[0m
Normally, the Linux Kernel talks straight to your hardware (video
card, keyboard, hard drives, etc), and any programs which run ask the
kernel to operate the hardware, like so:
+-----------+-----------+----+
| Process 1 | Process 2 | ...|
+-----------+-----------+----+
| Linux Kernel |
+----------------------------+
| Hardware |
+----------------------------+
The UML Kernel is different; instead of talking to the hardware, it
talks to a `real' Linux kernel (called the `host kernel' from now on),
like any other program. Programs can then run inside User-Mode Linux
as if they were running under a normal kernel, like so:
+----------------+
| Process 2 | ...|
+-----------+----------------+
| Process 1 | User-Mode Linux|
+----------------------------+
| Linux Kernel |
+----------------------------+
| Hardware |
+----------------------------+
[1m1.4. Why Would I Want UML?[0m
1. If UML crashes, your host kernel is still fine.
2. You can run a usermode kernel as a non-root user.
3. You can debug the UML like any normal process.
4. You can run gprof (profiling) and gcov (coverage testing).
5. You can play with your kernel without breaking things.
6. You can use it as a sandbox for testing new apps.
7. You can try new development kernels safely.
8. You can run different distributions simultaneously.
9. It's extremely fun.
[1m2. Compiling the kernel and modules[0m
[1m2.1. Compiling the kernel[0m
Compiling the user mode kernel is just like compiling any other
kernel. Let's go through the steps, using 2.4.0-prerelease (current
as of this writing) as an example:
1. Download the latest UML patch from
the download page
In this example, the file is uml-patch-2.4.0-prerelease.bz2.
2. Download the matching kernel from your favourite kernel mirror,
such as: http://ftp.ca.kernel.org/linux/kernel/
http://ftp.ca.kernel.org/linux/kernel/
.
3. Make a directory and unpack the kernel into it.
host%
mkdir ~/uml
host%
cd ~/uml
host%
tar -xjvf linux-2.4.0-prerelease.tar.bz2
4. Apply the patch using
host%
cd ~/uml/linux
host%
bzcat uml-patch-2.4.0-prerelease.bz2 | patch -p1
5. Run your favorite config; `make xconfig ARCH=um' is the most
convenient. `make config ARCH=um' and 'make menuconfig ARCH=um'
will work as well. The defaults will give you a useful kernel. If
you want to change something, go ahead, it probably won't hurt
anything.
Note: If the host is configured with a 2G/2G address space split
rather than the usual 3G/1G split, then the packaged UML binaries
will not run. They will immediately segfault. See ``UML on 2G/2G
hosts'' for the scoop on running UML on your system.
6. Finish with `make linux ARCH=um': the result is a file called
`linux' in the top directory of your source tree.
You may notice that the final binary is pretty large (many 10's of
megabytes for a debuggable UML). This is almost entirely symbol
information. The actual binary is comparable in size to a native
kernel. You can run that huge binary, and only the actual code and
data will be loaded into memory, so the symbols only consume disk
space unless you are running UML under gdb. You can strip UML:
host% strip linux
to see the true size of the UML kernel.
Make sure that you don't build this kernel in /usr/src/linux. On some
distributions, /usr/include/asm is a link into this pool. The user-
mode build changes the other end of that link, and things that include
stop compiling.
The sources are also available from cvs. You can browse
the CVS pool or access it
anonymously via
cvs -d:pserver:anonymous@www.user-mode-linux.org:/cvsroot/user-mode-linux
cvs command
If you get the CVS sources, you will have to check them out into an
empty directory. You will then have to copy each file into the
corresponding directory in the appropriate kernel pool.
If you don't have the latest kernel pool, you can get the
corresponding user-mode sources with
host% cvs co -r v_2_3_x linux
where 'x' is the version in your pool. Note that you will not get the
bug fixes and enhancements that have gone into subsequent releases.
If you build your own kernel, and want to boot it from one of the
filesystems distributed from this site, then, in nearly all cases,
devfs must be compiled into the kernel and mounted at boot time. The
exception is the tomsrtbt filesystem. For this, devfs must either not
be in the kernel at all, or "devfs=nomount" must be on the kernel
command line. Any disagreement between the kernel and the filesystem
being booted about whether devfs is being used will result in the boot
getting no further than single-user mode.
If you don't want to use devfs, you can remove the need for it from a
filesystem by copying /dev from someplace, making a bunch of /dev/ubd
devices:
UML#
for i in 0 1 2 3 4 5 6 7; do mknod ubd$i b 98 $[ $i * 16 ]; done
and changing /etc/fstab and /etc/inittab to refer to the non-devfs
devices.
[1m2.2. Compiling and installing kernel modules[0m
UML modules are built in the same way as the native kernel (with the
exception of the 'ARCH=um' that you always need for UML):
host% make modules ARCH=um
Any modules that you want to load into this kernel need to be built in
the user-mode pool. Modules from the native kernel won't work. If
you notice that the modules you get are much larger than they are on
the host, see the note above about the size of the final UML binary.
You can install them by using ftp or something to copy them into the
virtual machine and dropping them into /lib/modules/`uname -r`.
You can also get the kernel build process to install them as follows:
1. with the kernel not booted, mount the root filesystem in the top
level of the kernel pool:
host% mount root_fs mnt -o loop
2. run
host%
make modules_install INSTALL_MOD_PATH=`pwd`/mnt ARCH=um
3. unmount the filesystem
host% umount mnt
4. boot the kernel on it
If you can't mount the root filesystem on the host for some reason
(like it's a COW file), then an alternate approach is to mount the UML
kernel tree from the host into the UML with hostfs and run the modules_install inside
UML:
1. With UML booted, mount the host kernel tree inside UML at the same
location as on the host:
UML# mount none -t hostfs path to UML pool -o
path to UML pool
2. Run make modules_install:
UML# cd path to UML pool ; make modules_install
The depmod at the end may complain about unresolved symbols because
there is an incorrect or missing System.map installed in the UML
filesystem. This appears to be harmless. insmod or modprobe should
work fine at this point.
When the system is booted, you can use insmod as usual to get the
modules into the kernel. A number of things have been loaded into UML
as modules, especially filesystems and network protocols and filters,
so most symbols which need to be exported probably already are.
However, if you do find symbols that need exporting, let us
know, and
they'll be "taken care of".
If you try building an external module against a UML tree, you will
find that it doesn't compile because of missing includes. There are
less obvious problems with the CFLAGS that the module Makefile or
script provides which would make it not run even if it did build. To
get around this, you need to provide the same CFLAGS that the UML
kernel build uses.
A reasonably slick way of getting the UML CFLAGS is
cd uml-tree ; make script 'SCRIPT=@echo $(CFLAGS)' ARCH=um
If the module build process has something that looks like
$(CC) $(CFLAGS) file
then you can define CFLAGS in a script like this
CFLAGS=`cd uml-tree ; make script 'SCRIPT=@echo $(CFLAGS)' ARCH=um`
and like this in a Makefile
CFLAGS=$(shell cd uml-tree ; make script 'SCRIPT=@echo
$$(CFLAGS)' ARCH=um)
[1m2.3. Compiling and installing uml_utilities[0m
Many features of the UML kernel require a user-space helper program,
so a uml_utilities package is distributed separately from the kernel
patch which provides these helpers. Included within this is:
+o port-helper - Used by consoles which connect to xterms or ports
+o tunctl - Configuration tool to create and delete tap devices
+o uml_net - Setuid binary for automatic tap device configuration
+o uml_switch - User-space virtual switch required for daemon
transport
The uml_utilities tree is compiled with:
host#
make && make install
Note that UML kernel patches may require a specific version of the
uml_utilities distribution. If you don't keep up with the mailing
lists, ensure that you have the latest release of uml_utilities if you
are experiencing problems with your UML kernel, particularly when
dealing with consoles or command-line switches to the helper programs
[1m3. Running UML and logging in[0m
[1m3.1. Running UML[0m
It runs on 2.2.15 or later, and all 2.4 and 2.6 kernels.
Booting UML is straightforward. Simply run 'linux': it will try to
mount the file `root_fs' in the current directory. You do not need to
run it as root. If your root filesystem is not named `root_fs', then
you need to put a `ubd0=root_fs_whatever' switch on the linux command
line.
You will need a filesystem to boot UML from. There are a number
available for download from here . There are also several tools
which can be
used to generate UML-compatible filesystem images from media.
The kernel will boot up and present you with a login prompt.
Note: If the host is configured with a 2G/2G address space split
rather than the usual 3G/1G split, then the packaged UML binaries will
not run. They will immediately segfault. See ``UML on 2G/2G hosts''
for the scoop on running UML on your system.
[1m3.2. Logging in[0m
The prepackaged filesystems have a root account with password 'root'
and a user account with password 'user'. The login banner will
generally tell you how to log in. So, you log in and you will find
yourself inside a little virtual machine. Our filesystems have a
variety of commands and utilities installed (and it is fairly easy to
add more), so you will have a lot of tools with which to poke around
the system.
There are a couple of other ways to log in:
+o On a virtual console
Each virtual console that is configured (i.e. the device exists in
/dev and /etc/inittab runs a getty on it) will come up in its own
xterm. If you get tired of the xterms, read ``Setting up serial
lines and consoles'' to see how to attach the consoles to
something else, like host ptys.
+o Over the serial line
In the boot output, find a line that looks like:
serial line 0 assigned pty /dev/ptyp1
Attach your favorite terminal program to the corresponding tty. I.e.
for minicom, the command would be
host% minicom -o -p /dev/ttyp1
+o Over the net
If the network is running, then you can telnet to the virtual
machine and log in to it. See ``Setting up the network'' to learn
about setting up a virtual network.
When you're done using it, run halt, and the kernel will bring itself
down and the process will exit.
[1m3.3. Examples[0m
Here are some examples of UML in action:
+o A login session
+o A virtual network
[1m4. UML on 2G/2G hosts[0m
[1m4.1. Introduction[0m
Most Linux machines are configured so that the kernel occupies the
upper 1G (0xc0000000 - 0xffffffff) of the 4G address space and
processes use the lower 3G (0x00000000 - 0xbfffffff). However, some
machine are configured with a 2G/2G split, with the kernel occupying
the upper 2G (0x80000000 - 0xffffffff) and processes using the lower
2G (0x00000000 - 0x7fffffff).
[1m4.2. The problem[0m
The prebuilt UML binaries on this site will not run on 2G/2G hosts
because UML occupies the upper .5G of the 3G process address space
(0xa0000000 - 0xbfffffff). Obviously, on 2G/2G hosts, this is right
in the middle of the kernel address space, so UML won't even load - it
will immediately segfault.
[1m4.3. The solution[0m
The fix for this is to rebuild UML from source after enabling
CONFIG_HOST_2G_2G (under 'General Setup'). This will cause UML to
load itself in the top .5G of that smaller process address space,
where it will run fine. See ``Compiling the kernel and modules'' if
you need help building UML from source.
[1m5. Setting up serial lines and consoles[0m
It is possible to attach UML serial lines and consoles to many types
of host I/O channels by specifying them on the command line.
You can attach them to host ptys, ttys, file descriptors, and ports.
This allows you to do things like
+o have a UML console appear on an unused host console,
+o hook two virtual machines together by having one attach to a pty
and having the other attach to the corresponding tty
+o make a virtual machine accessible from the net by attaching a
console to a port on the host.
The general format of the command line option is device=channel.
[1m5.1. Specifying the device[0m
Devices are specified with "con" or "ssl" (console or serial line,
respectively), optionally with a device number if you are talking
about a specific device.
Using just "con" or "ssl" describes all of the consoles or serial
lines. If you want to talk about console #3 or serial line #10, they
would be "con3" and "ssl10", respectively.
A specific device name will override a less general "con=" or "ssl=".
So, for example, you can assign a pty to each of the serial lines
except for the first two like this:
ssl=pty ssl0=tty:/dev/tty0 ssl1=tty:/dev/tty1
The specificity of the device name is all that matters; order on the
command line is irrelevant.
[1m5.2. Specifying the channel[0m
There are a number of different types of channels to attach a UML
device to, each with a different way of specifying exactly what to
attach to.
+o pseudo-terminals - device=pty pts terminals - device=pts
This will cause UML to allocate a free host pseudo-terminal for the
device. The terminal that it got will be announced in the boot
log. You access it by attaching a terminal program to the
corresponding tty:
+o screen /dev/pts/n
+o screen /dev/ttyxx
+o minicom -o -p /dev/ttyxx - minicom seems not able to handle pts
devices
+o kermit - start it up, 'open' the device, then 'connect'
+o terminals - device=tty:tty device file
This will make UML attach the device to the specified tty (i.e
con1=tty:/dev/tty3
will attach UML's console 1 to the host's /dev/tty3). If the tty that
you specify is the slave end of a tty/pty pair, something else must
have already opened the corresponding pty in order for this to work.
+o xterms - device=xterm
UML will run an xterm and the device will be attached to it.
+o Port - device=port:port number
This will attach the UML devices to the specified host port.
Attaching console 1 to the host's port 9000 would be done like
this:
con1=port:9000
Attaching all the serial lines to that port would be done similarly:
ssl=port:9000
You access these devices by telnetting to that port. Each active tel-
net session gets a different device. If there are more telnets to a
port than UML devices attached to it, then the extra telnet sessions
will block until an existing telnet detaches, or until another device
becomes active (i.e. by being activated in /etc/inittab).
This channel has the advantage that you can both attach multiple UML
devices to it and know how to access them without reading the UML boot
log. It is also unique in allowing access to a UML from remote
machines without requiring that the UML be networked. This could be
useful in allowing public access to UMLs because they would be
accessible from the net, but wouldn't need any kind of network
filtering or access control because they would have no network access.
If you attach the main console to a portal, then the UML boot will
appear to hang. In reality, it's waiting for a telnet to connect, at
which point the boot will proceed.
+o already-existing file descriptors - device=file descriptor
If you set up a file descriptor on the UML command line, you can
attach a UML device to it. This is most commonly used to put the
main console back on stdin and stdout after assigning all the other
consoles to something else:
con0=fd:0,fd:1 con=pts
+o Nothing - device=null
This allows the device to be opened, in contrast to 'none', but
reads will block, and writes will succeed and the data will be
thrown out.
+o None - device=none
This causes the device to disappear. If you are using devfs, the
device will not appear in /dev. If not, then attempts to open it
will return -ENODEV.
You can also specify different input and output channels for a device
by putting a comma between them:
ssl3=tty:/dev/tty2,xterm
will cause serial line 3 to accept input on the host's /dev/tty3 and
display output on an xterm. That's a silly example - the most common
use of this syntax is to reattach the main console to stdin and stdout
as shown above.
If you decide to move the main console away from stdin/stdout, the
initial boot output will appear in the terminal that you're running
UML in. However, once the console driver has been officially
initialized, then the boot output will start appearing wherever you
specified that console 0 should be. That device will receive all
subsequent output.
[1m5.3. Examples[0m
There are a number of interesting things you can do with this
capability.
First, this is how you get rid of those bleeding console xterms by
attaching them to host ptys:
con=pty con0=fd:0,fd:1
This will make a UML console take over an unused host virtual console,
so that when you switch to it, you will see the UML login prompt
rather than the host login prompt:
con1=tty:/dev/tty6
You can attach two virtual machines together with what amounts to a
serial line as follows:
Run one UML with a serial line attached to a pty -
ssl1=pty
Look at the boot log to see what pty it got (this example will assume
that it got /dev/ptyp1).
Boot the other UML with a serial line attached to the corresponding
tty -
ssl1=tty:/dev/ttyp1
Log in, make sure that it has no getty on that serial line, attach a
terminal program like minicom to it, and you should see the login
prompt of the other virtual machine.
[1m6. Setting up the network[0m
This page describes how to set up the various transports and to
provide a UML instance with network access to the host, other machines
on the local net, and the rest of the net.
As of 2.4.5, UML networking has been completely redone to make it much
easier to set up, fix bugs, and add new features.
There is a new helper, uml_net, which does the host setup that
requires root privileges.
There are currently five transport types available for a UML virtual
machine to exchange packets with other hosts:
+o ethertap
+o TUN/TAP
+o Multicast
+o a switch daemon
+o slip
+o slirp
+o pcap
The TUN/TAP, ethertap, slip, and slirp transports allow a UML
instance to exchange packets with the host. They may be directed
to the host or the host may just act as a router to provide access
to other physical or virtual machines.
The pcap transport is a synthetic read-only interface, using the
libpcap binary to collect packets from interfaces on the host and
filter them. This is useful for building preconfigured traffic
monitors or sniffers.
The daemon and multicast transports provide a completely virtual
network to other virtual machines. This network is completely
disconnected from the physical network unless one of the virtual
machines on it is acting as a gateway.
With so many host transports, which one should you use? Here's when
you should use each one:
+o ethertap - if you want access to the host networking and it is
running 2.2
+o TUN/TAP - if you want access to the host networking and it is
running 2.4. Also, the TUN/TAP transport is able to use a
preconfigured device, allowing it to avoid using the setuid uml_net
helper, which is a security advantage.
+o Multicast - if you want a purely virtual network and you don't want
to set up anything but the UML
+o a switch daemon - if you want a purely virtual network and you
don't mind running the daemon in order to get somewhat better
performance
+o slip - there is no particular reason to run the slip backend unless
ethertap and TUN/TAP are just not available for some reason
+o slirp - if you don't have root access on the host to setup
networking, or if you don't want to allocate an IP to your UML
+o pcap - not much use for actual network connectivity, but great for
monitoring traffic on the host
Ethertap is available on 2.4 and works fine. TUN/TAP is preferred
to it because it has better performance and ethertap is officially
considered obsolete in 2.4. Also, the root helper only needs to
run occasionally for TUN/TAP, rather than handling every packet, as
it does with ethertap. This is a slight security advantage since
it provides fewer opportunities for a nasty UML user to somehow
exploit the helper's root privileges.
[1m6.1. General setup[0m
First, you must have the virtual network enabled in your UML. If are
running a prebuilt kernel from this site, everything is already
enabled. If you build the kernel yourself, under the "Network device
support" menu, enable "Network device support", and then the three
transports.
The next step is to provide a network device to the virtual machine.
This is done by describing it on the kernel command line.
The general format is
eth = ,
For example, a virtual ethernet device may be attached to a host
ethertap device as follows:
eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
This sets up eth0 inside the virtual machine to attach itself to the
host /dev/tap0, assigns it an ethernet address, and assigns the host
tap0 interface an IP address.
Note that the IP address you assign to the host end of the tap device
must be different than the IP you assign to the eth device inside UML.
If you are short on IPs and don't want to comsume two per UML, then
you can reuse the host's eth IP address for the host ends of the tap
devices. Internally, the UMLs must still get unique IPs for their eth
devices. You can also give the UMLs non-routable IPs (192.168.x.x or
10.x.x.x) and have the host masquerade them. This will let outgoing
connections work, but incoming connections won't without more work,
such as port forwarding from the host.
Also note that when you configure the host side of an interface, it is
only acting as a gateway. It will respond to pings sent to it
locally, but is not useful to do that since it's a host interface.
You are not talking to the UML when you ping that interface and get a
response.
You can also add devices to a UML and remove them at runtime. See the
``The Management Console'' page for details.
The sections below describe this in more detail.
Once you've decided how you're going to set up the devices, you boot
UML, log in, configure the UML side of the devices, and set up routes
to the outside world. At that point, you will be able to talk to any
other machines, physical or virtual, on the net.
If ifconfig inside UML fails and the network refuses to come up, run
tell you what went wrong.
[1m6.2. Userspace daemons[0m
You will likely need the setuid helper, or the switch daemon, or both.
They are both installed with the RPM and deb, so if you've installed
either, you can skip the rest of this section.
If not, then you need to check them out of CVS, build them, and
install them. The helper is uml_net, in CVS /tools/uml_net, and the
daemon is uml_switch, in CVS /tools/uml_router. They are both built
with a plain 'make'. Both need to be installed in a directory that's
in your path - /usr/bin is recommend. On top of that, uml_net needs
to be setuid root.
[1m6.3. Specifying ethernet addresses[0m
Below, you will see that the TUN/TAP, ethertap, and daemon interfaces
allow you to specify hardware addresses for the virtual ethernet
devices. This is generally not necessary. If you don't have a
specific reason to do it, you probably shouldn't. If one is not
specified on the command line, the driver will assign one based on the
device IP address. It will provide the address fe:fd:nn:nn:nn:nn
where nn.nn.nn.nn is the device IP address. This is nearly always
sufficient to guarantee a unique hardware address for the device. A
couple of exceptions are:
+o Another set of virtual ethernet devices are on the same network and
they are assigned hardware addresses using a different scheme which
may conflict with the UML IP address-based scheme
+o You aren't going to use the device for IP networking, so you don't
assign the device an IP address
If you let the driver provide the hardware address, you should make
sure that the device IP address is known before the interface is
brought up. So, inside UML, this will guarantee that:
UML#
ifconfig eth0 192.168.0.250 up
If you decide to assign the hardware address yourself, make sure that
the first byte of the address is even. Addresses with an odd first
byte are broadcast addresses, which you don't want assigned to a
device.
[1m6.4. UML interface setup[0m
Once the network devices have been described on the command line, you
should boot UML and log in.
The first thing to do is bring the interface up:
UML# ifconfig ethn ip-address up
You should be able to ping the host at this point.
To reach the rest of the world, you should set a default route to the
host:
UML# route add default gw host ip
Again, with host ip of 192.168.0.4:
UML# route add default gw 192.168.0.4
This page used to recommend setting a network route to your local net.
This is wrong, because it will cause UML to try to figure out hardware
addresses of the local machines by arping on the interface to the
host. Since that interface is basically a single strand of ethernet
with two nodes on it (UML and the host) and arp requests don't cross
networks, they will fail to elicit any responses. So, what you want
is for UML to just blindly throw all packets at the host and let it
figure out what to do with them, which is what leaving out the network
route and adding the default route does.
Note: If you can't communicate with other hosts on your physical
ethernet, it's probably because of a network route that's
automatically set up. If you run 'route -n' and see a route that
looks like this:
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
with a mask that's not 255.255.255.255, then replace it with a route
to your host:
UML#
route del -net 192.168.0.0 dev eth0 netmask 255.255.255.0
UML#
route add -host 192.168.0.4 dev eth0
This, plus the default route to the host, will allow UML to exchange
packets with any machine on your ethernet.
[1m6.5. Multicast[0m
The simplest way to set up a virtual network between multiple UMLs is
to use the mcast transport. This was written by Harald Welte and is
present in UML version 2.4.5-5um and later. Your system must have
multicast enabled in the kernel and there must be a multicast-capable
network device on the host. Normally, this is eth0, but if there is
no ethernet card on the host, then you will likely get strange error
messages when you bring the device up inside UML.
To use it, run two UMLs with
eth0=mcast
on their command lines. Log in, configure the ethernet device in each
machine with different IP addresses:
UML1# ifconfig eth0 192.168.0.254
UML2# ifconfig eth0 192.168.0.253
and they should be able to talk to each other.
The full set of command line options for this transport are
ethn=mcast,ethernet address,multicast
address,multicast port,ttl
Harald's original README is here and explains these in detail, as well as
some other issues.
[1m6.6. TUN/TAP with the uml_net helper[0m
TUN/TAP is the preferred mechanism on 2.4 to exchange packets with the
host. The TUN/TAP backend has been in UML since 2.4.9-3um.
The easiest way to get up and running is to let the setuid uml_net
helper do the host setup for you. This involves insmod-ing the tun.o
module if necessary, configuring the device, and setting up IP
forwarding, routing, and proxy arp. If you are new to UML networking,
do this first. If you're concerned about the security implications of
the setuid helper, use it to get up and running, then read the next
section to see how to have UML use a preconfigured tap device, which
avoids the use of uml_net.
If you specify an IP address for the host side of the device, the
uml_net helper will do all necessary setup on the host - the only
requirement is that TUN/TAP be available, either built in to the host
kernel or as the tun.o module.
The format of the command line switch to attach a device to a TUN/TAP
device is
eth =tuntap,,,
For example, this argument will attach the UML's eth0 to the next
available tap device, assign the IP address 192.168.0.254 to the host
side of the tap device, and assign an ethernet address to it based on
the IP address assigned to it by ifconfig inside UML.
eth0=tuntap,,,192.168.0.254
If you using the uml_net helper to set up the host side of the
networking, as in this example, note that changing the UML IP address
will cause uml_net to change the host routing and arping to match.
This is one reason you should not be using uml_net if there is any
possibility that the user inside the UML may be unfriendly. This
feature is convenient, but can be used to make the UML pretend to be
something like your name server or mail server, and the host will
steal packets intended for those servers and forward them to the UML.
See the next section for setting up networking in a secure manner.
There are a couple potential problems with running the TUN/TAP
transport on a 2.4 host kernel
+o TUN/TAP seems not to work on 2.4.3 and earlier. Upgrade the host
kernel or use the ethertap transport.
+o With an upgraded kernel, TUN/TAP may fail with
File descriptor in bad state
This is due to a header mismatch between the upgraded kernel and the
kernel that was originally installed on the machine. The fix is to
make sure that /usr/src/linux points to the headers for the running
kernel.
These were pointed out by Tim Robinson in
name="this uml-
user post"> .
[1m6.7. TUN/TAP with a preconfigured tap device[0m
If you prefer not to have UML use uml_net (which is somewhat
insecure), with UML 2.4.17-11, you can set up a TUN/TAP device
beforehand. The setup needs to be done as root, but once that's done,
there is no need for root assistance. Setting up the device is done
as follows:
+o Create the device with tunctl (available from the UML utilities
tarball)
host# tunctl -u uid
where uid is the user id or username that UML will be run as. This
will tell you what device was created.
+o Configure the device IP (change IP addresses and device name to
suit)
host# ifconfig tap0 192.168.0.254 up
+o Set up routing and arping if desired - this is my recipe, there are
other ways of doing the same thing
host#
bash -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
host#
route add -host 192.168.0.253 dev tap0
host#
bash -c 'echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp'
host#
arp -Ds 192.168.0.253 eth0 pub
Note that this must be done every time the host boots - this configu-
ration is not stored across host reboots. So, it's probably a good
idea to stick it in an rc file. An even better idea would be a little
utility which reads the information from a config file and sets up
devices at boot time.
+o Rather than using up two IPs and ARPing for one of them, you can
also provide direct access to your LAN by the UML by using a
bridge.
host#
brctl addbr br0
host#
ifconfig eth0 0.0.0.0 promisc up
host#
ifconfig tap0 0.0.0.0 promisc up
host#
ifconfig br0 192.168.0.1 netmask 255.255.255.0 up
host#
brctl stp br0 off
host#
brctl setfd br0 1
host#
brctl sethello br0 1
host#
brctl addif br0 eth0
host#
brctl addif br0 tap0
Note that 'br0' should be setup using ifconfig with the existing IP
address of eth0, as eth0 no longer has its own IP.
+o
Also, the /dev/net/tun device must be writable by the user running
UML in order for the UML to use the device that's been configured
for it. The simplest thing to do is
host# chmod 666 /dev/net/tun
Making it world-writeable looks bad, but it seems not to be
exploitable as a security hole. However, it does allow anyone to cre-
ate useless tap devices (useless because they can't configure them),
which is a DOS attack. A somewhat more secure alternative would to be
to create a group containing all the users who have preconfigured tap
devices and chgrp /dev/net/tun to that group with mode 664 or 660.
+o Once the device is set up, run UML with
eth0=tuntap,devicename
i.e.
eth0=tuntap,tap0
on the command line (or do it with the mconsole config command).
+o Bring the eth device up in UML and you're in business.
If you don't want that tap device any more, you can make it non-
persistent with
host# tunctl -d tap device
Finally, tunctl has a -b (for brief mode) switch which causes it to
output only the name of the tap device it created. This makes it
suitable for capture by a script:
host# TAP=`tunctl -u 1000 -b`
[1m6.8. Ethertap[0m
Ethertap is the general mechanism on 2.2 for userspace processes to
exchange packets with the kernel.
To use this transport, you need to describe the virtual network device
on the UML command line. The general format for this is
eth =ethertap, , ,
So, the previous example
eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
attaches the UML eth0 device to the host /dev/tap0, assigns it the
ethernet address fe:fd:0:0:0:1, and assigns the IP address
192.168.0.254 to the host side of the tap device.
The tap device is mandatory, but the others are optional. If the
ethernet address is omitted, one will be assigned to it.
The presence of the tap IP address will cause the helper to run and do
whatever host setup is needed to allow the virtual machine to
communicate with the outside world. If you're not sure you know what
you're doing, this is the way to go.
If it is absent, then you must configure the tap device and whatever
arping and routing you will need on the host. However, even in this
case, the uml_net helper still needs to be in your path and it must be
setuid root if you're not running UML as root. This is because the
tap device doesn't support SIGIO, which UML needs in order to use
something as a source of input. So, the helper is used as a
convenient asynchronous IO thread.
If you're using the uml_net helper, you can ignore the following host
setup - uml_net will do it for you. You just need to make sure you
have ethertap available, either built in to the host kernel or
available as a module.
If you want to set things up yourself, you need to make sure that the
appropriate /dev entry exists. If it doesn't, become root and create
it as follows (the $[ ... ] is bash syntax for adding 16 to the minor
number) :
mknod /dev/tap c 36 $[ + 16 ]
For example, this is how to create /dev/tap0:
mknod /dev/tap0 c 36 $[ 0 + 16 ]
You also need to make sure that the host kernel has ethertap support.
If ethertap is enabled as a module, you apparently need to insmod
ethertap once for each ethertap device you want to enable. So,
host#
insmod ethertap
will give you the tap0 interface. To get the tap1 interface, you need
to run
host#
insmod ethertap unit=1 -o ethertap1
[1m6.9. The switch daemon[0m
[1mNote[22m: This is the daemon formerly known as uml_router, but which was
renamed so the network weenies of the world would stop growling at me.
The switch daemon, uml_switch, provides a mechanism for creating a
totally virtual network. By default, it provides no connection to the
host network (but see -tap, below).
The first thing you need to do is run the daemon. Running it with no
arguments will make it listen on a default unix domain socket.
If you want it to listen on a different socket, use
-unix socket
If you want it to act as a hub rather than a switch, use
-hub
If you're planning on putting it in hub mode so you can sniff UML
traffic from a tap device on the host, it appears that you need to
assign the tap an IP address before you'll see any packets on it.
If you want the switch to be connected to host networking (allowing
the umls to get access to the outside world through the host), use
-tap tap0
Note that the tap device must be preconfigured (see "TUN/TAP with a
preconfigured tap device", above). If you're using a different tap
device than tap0, specify that instead of tap0.
uml_switch can be backgrounded as follows
host%
uml_switch [ options ] < /dev/null > /dev/null
The reason it doesn't background by default is that it listens to
stdin for EOF. When it sees that, it exits.
The general format of the kernel command line switch is
ethn=daemon,ethernet address,socket type,socket
You can leave off everything except the 'daemon'. You only need to
specify the ethernet address if the one that will be assigned to it
isn't acceptable for some reason. The rest of the arguments describe
how to communicate with the daemon. You should only specify them if
you told the daemon to use different sockets than the default. So, if
you ran the daemon with no arguments, running the UML on the same
machine with
eth0=daemon
will cause the eth0 driver to attach itself to the daemon correctly.
The socket argument is the filename of a Unix domain socket which is
used for communications between uml_switch and the UMLs on its net-
work. If you do specify a different socket from the default, which
you will need to do if you want multiple, separate uml_switch networks
on the host, you need to make sure that you name the same path for the
socket on both the uml_switch and UML command lines.
Currently the only supported value for the socket type is "unix".
[1m6.10. Slip[0m
Slip is another, less general, mechanism for a process to communicate
with the host networking. In contrast to the ethertap interface,
which exchanges ethernet frames with the host and can be used to
transport any higher-level protocol, it can only be used to transport
IP.
The general format of the command line switch is
ethn=slip,slip IP
The slip IP argument is the IP address that will be assigned to the
host end of the slip device. If it is specified, the helper will run
and will set up the host so that the virtual machine can reach it and
the rest of the network.
There are some oddities with this interface that you should be aware
of. You should only specify one slip device on a given virtual
machine, and its name inside UML will be 'umn', not 'eth0' or whatever
you specified on the command line. These problems will be fixed at
some point.
[1m6.11. Slirp[0m
slirp uses an external program, usually /usr/bin/slirp, to provide IP
only networking connectivity through the host. This is similar to IP
masquerading with a firewall, although the translation is performed in
user-space, rather than by the kernel. As slirp does not set up any
interfaces on the host, or changes routing, slirp does not require
root access or setuid binaries on the host.
The general format of the command line switch for slirp is:
ethn=slirp,ethernet address,slirp path
The ethernet address is optional, as UML will set up the interface
with an ethernet address based upon the initial IP address of the
interface. The slirp path is generally /usr/bin/slirp, although it
will depend on distribution.
The slirp program can have a number of options passed to the command
line and we can't add them to the UML command line, as they will be
parsed incorrectly. Instead, a wrapper shell script can be written or
the options inserted into the /.slirprc file. More information on
all of the slirp options can be found in its man pages.
The eth0 interface on UML should be set up with the IP 10.2.0.15,
although you can use anything as long as it is not used by a network
you will be connecting to. The default route on UML should be set to
use
UML#
route add default dev eth0
slirp provides a number of useful IP addresses which can be used by
UML, such as 10.0.2.3 which is an alias for the DNS server specified
in /etc/resolv.conf on the host or the IP given in the 'dns' option
for slirp.
Even with a baudrate setting higher than 115200, the slirp connection
is limited to 115200. If you need it to go faster, the slirp binary
needs to be compiled with FULL_BOLT defined in config.h.
[1m6.12. pcap[0m
The pcap transport is attached to a UML ethernet device on the command
line or with uml_mconsole with the following syntax:
ethn=pcap,host interface,filter
expression,option1,option2
The expression and options are optional.
The interface is whatever network device on the host you want to
sniff. The expression is a pcap filter expression, which is also what
tcpdump uses, so if you know how to specify tcpdump filters, you will
use the same expressions here. The options are up to two of
'promisc', control whether pcap puts the host interface into
promiscuous mode. 'optimize' and 'nooptimize' control whether the pcap
expression optimizer is used.
Example:
eth0=pcap,eth0,tcp
eth1=pcap,eth0,!tcp
will cause the UML eth0 to emit all tcp packets on the host eth0 and
the UML eth1 to emit all non-tcp packets on the host eth0.
[1m6.13. Setting up the host yourself[0m
If you don't specify an address for the host side of the ethertap or
slip device, UML won't do any setup on the host. So this is what is
needed to get things working (the examples use a host-side IP of
192.168.0.251 and a UML-side IP of 192.168.0.250 - adjust to suit your
own network):
+o The device needs to be configured with its IP address. Tap devices
are also configured with an mtu of 1484. Slip devices are
configured with a point-to-point address pointing at the UML ip
address.
host# ifconfig tap0 arp mtu 1484 192.168.0.251 up
host#
ifconfig sl0 192.168.0.251 pointopoint 192.168.0.250 up
+o If a tap device is being set up, a route is set to the UML IP.
UML# route add -host 192.168.0.250 gw 192.168.0.251
+o To allow other hosts on your network to see the virtual machine,
proxy arp is set up for it.
host# arp -Ds 192.168.0.250 eth0 pub
+o Finally, the host is set up to route packets.
host# echo 1 > /proc/sys/net/ipv4/ip_forward
[1m7. Sharing Filesystems between Virtual Machines[0m
[1m7.1. A warning[0m
Don't attempt to share filesystems simply by booting two UMLs from the
same file. That's the same thing as booting two physical machines
from a shared disk. It will result in filesystem corruption.
[1m7.2. Using layered block devices[0m
The way to share a filesystem between two virtual machines is to use
the copy-on-write (COW) layering capability of the ubd block driver.
As of 2.4.6-2um, the driver supports layering a read-write private
device over a read-only shared device. A machine's writes are stored
in the private device, while reads come from either device - the
private one if the requested block is valid in it, the shared one if
not. Using this scheme, the majority of data which is unchanged is
shared between an arbitrary number of virtual machines, each of which
has a much smaller file containing the changes that it has made. With
a large number of UMLs booting from a large root filesystem, this
leads to a huge disk space saving. It will also help performance,
since the host will be able to cache the shared data using a much
smaller amount of memory, so UML disk requests will be served from the
host's memory rather than its disks.
To add a copy-on-write layer to an existing block device file, simply
add the name of the COW file to the appropriate ubd switch:
ubd0=root_fs_cow,root_fs_debian_22
where 'root_fs_cow' is the private COW file and 'root_fs_debian_22' is
the existing shared filesystem. The COW file need not exist. If it
doesn't, the driver will create and initialize it. Once the COW file
has been initialized, it can be used on its own on the command line:
ubd0=root_fs_cow
The name of the backing file is stored in the COW file header, so it
would be redundant to continue specifying it on the command line.
[1m7.3. Note![0m
When checking the size of the COW file in order to see the gobs of
space that you're saving, make sure you use 'ls -ls' to see the actual
disk consumption rather than the length of the file. The COW file is
sparse, so the length will be very different from the disk usage.
Here is a 'ls -l' of a COW file and backing file from one boot and
shutdown:
host% ls -l cow.debian debian2.2
-rw-r--r-- 1 jdike jdike 492504064 Aug 6 21:16 cow.debian
-rwxrw-rw- 1 jdike jdike 537919488 Aug 6 20:42 debian2.2
Doesn't look like much saved space, does it? Well, here's 'ls -ls':
host% ls -ls cow.debian debian2.2
880 -rw-r--r-- 1 jdike jdike 492504064 Aug 6 21:16 cow.debian
525832 -rwxrw-rw- 1 jdike jdike 537919488 Aug 6 20:42 debian2.2
Now, you can see that the COW file has less than a meg of disk, rather
than 492 meg.
[1m7.4. Another warning[0m
Once a filesystem is being used as a readonly backing file for a COW
file, do not boot directly from it or modify it in any way. Doing so
will invalidate any COW files that are using it. The mtime and size
of the backing file are stored in the COW file header at its creation,
and they must continue to match. If they don't, the driver will
refuse to use the COW file.
If you attempt to evade this restriction by changing either the
backing file or the COW header by hand, you will get a corrupted
filesystem.
Among other things, this means that upgrading the distribution in a
backing file and expecting that all of the COW files using it will see
the upgrade will not work.
[1m7.5. Moving a backing file[0m
Because UML stores the backing file name and its mtime in the COW
header, if you move the backing file, that information becomes
invalid. So, the procedure for moving a backing file is
+o Move it in a way that preserves timestamps. Usually, this is a
"-p" switch. "cp -a" works because "-a" implies "-p".
+o Update the COW header by booting UML on it, specifying both the COW
file and the new location of the backing file
host% ubd0=COW file,new backing file
location
UML will notice the mismatch between the command line and COW header,
check the size and mtime of the new backing file path, and update the
COW header to reflect it if it checks out.
If you forget to preserve the timestamps when you move the backing
file, you can fix the mtime by hand as follows
host%
mtime=whatever UML says mtime should be ; \
touch --date="`date -d 1970-01-01\ UTC\ $mtime\ seconds`" backing file
Note that if you do this on a backing file that has truly been
changed, and not just moved, then you will get file corruption and you
will lose the filesystem.
[1m7.6. uml_moo : Merging a COW file with its backing file[0m
Depending on how you use UML and COW devices, it may be advisable to
merge the changes in the COW file into the backing file every once in
a while.
The utility that does this is uml_moo. Its usage is
host% uml_moo COW file new backing file
There's no need to specify the backing file since that information is
already in the COW file header. If you're paranoid, boot the new
merged file, and if you're happy with it, move it over the old backing
file.
uml_moo creates a new backing file by default as a safety measure. It
also has a destructive merge option which will merge the COW file
directly into its current backing file. This is really only usable
when the backing file only has one COW file associated with it. If
there are multiple COWs associated with a backing file, a -d merge of
one of them will invalidate all of the others. However, it is
convenient if you're short of disk space, and it should also be
noticably faster than a non-destructive merge. This usage is
host% uml_moo -d COW file
uml_moo is installed with the UML deb and RPM. If you didn't install
UML from one of those packages, you can also get it from the UML
utilities tar file in tools/moo.
[1m7.7. uml_mkcow : Create a new COW file[0m
The normal way to create a COW file is to specify a non-existant COW
file on the UML command line, and let UML create it for you. However,
sometimes you want a new COW file, and you don't want to boot UML in
order to get it. This can be done with uml_mkcow, which is a little
standalone utility by Steve Schnepp.
The standard usage is
host% uml_mkcow new COW file existing
backing file
If you want to destroy an existing COW file, then there is a -f switch
to force the overwriting of the old COW file
host% uml_mkcow -f existing COW file existing
backing file
uml_mkcow is available from the UML utilities tar file in
tools/moo.
[1m8. Creating filesystems[0m
You may want to create and mount new UML filesystems, either because
your root filesystem isn't large enough or because you want to use a
filesystem other than ext2.
This was written on the occasion of reiserfs being included in the
2.4.1 kernel pool, and therefore the 2.4.1 UML, so the examples will
talk about reiserfs. This information is generic, and the examples
should be easy to translate to the filesystem of your choice.
[1m8.1. Create the filesystem file[0m
dd is your friend. All you need to do is tell dd to create an empty
file of the appropriate size. I usually make it sparse to save time
and to avoid allocating disk space until it's actually used. For
example, the following command will create a sparse 100 meg file full
of zeroes.
host%
dd if=/dev/zero of=new_filesystem seek=100 count=1 bs=1M
[1m8.2. Assign the file to a UML device[0m
Add an argument like the following to the UML command line:
ubd4=new_filesystem
making sure that you use an unassigned ubd device number.
[1m8.3. Creating and mounting the filesystem[0m
Make sure that the filesystem is available, either by being built into
the kernel, or available as a module, then boot up UML and log in. If
the root filesystem doesn't have the filesystem utilities (mkfs, fsck,
etc), then get them into UML by way of the net or hostfs.
Make the new filesystem on the device assigned to the new file:
host# mkreiserfs /dev/ubd/4
<----------- MKREISERFSv2 ----------->
ReiserFS version 3.6.25
Block size 4096 bytes
Block count 25856
Used blocks 8212
Journal - 8192 blocks (18-8209), journal header is in block 8210
Bitmaps: 17
Root block 8211
Hash function "r5"
ATTENTION: ALL DATA WILL BE LOST ON '/dev/ubd/4'! (y/n)y
journal size 8192 (from 18)
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..done.
Now, mount it:
UML#
mount /dev/ubd/4 /mnt
and you're in business.
[1m9. Host file access[0m
If you want to access files on the host machine from inside UML, you
can treat it as a separate machine and either nfs mount directories
from the host or copy files into the virtual machine with scp or rcp.
However, since UML is running on the the host, it can access those
files just like any other process and make them available inside the
virtual machine without needing to use the network.
This is now possible with the hostfs virtual filesystem. With it, you
can mount a host directory into the UML filesystem and access the
files contained in it just as you would on the host.
Note that hostfs is currently not available on 2.5. The reason is
that there was an fs.h rework early in 2.5 which required filesystem
changes, and I haven't got around to updating hostfs to those changes.
[1m9.1. Using hostfs[0m
To begin with, make sure that hostfs is available inside the virtual
machine with
UML# cat /proc/filesystems
hostfs should be listed. If it's not, either rebuild the kernel with
hostfs configured into it or make sure that hostfs is built as a mod-
ule and available inside the virtual machine, and insmod it.
Now all you need to do is run mount:
UML# mount none /mnt/host -t hostfs
will mount the host's / on the virtual machine's /mnt/host.
If you don't want to mount the host root directory, then you can
specify a subdirectory to mount with the -o switch to mount:
UML# mount none /mnt/home -t hostfs -o /home
will mount the hosts's /home on the virtual machine's /mnt/home.
[1m9.2. hostfs command line options[0m
There is a hostfs option available on the UML command line which can
be used confine all hostfs mounts to a host directory hierarchy or to
prevent a hostfs user from destroying data on the host. The format is
hostfs=directory,options
The only option available at present is 'append', which forces all
files to be opened in append mode and disallows any deletion of files.
To specify append mode without confining hostfs to a host directory,
just leave out the directory name so that the argument begins with a
comma:
hostfs=,append
[1m9.3. hostfs as the root filesystem[0m
It's possible to boot from a directory hierarchy on the host using
hostfs rather than using the standard filesystem in a file.
To start, you need that hierarchy. The easiest way is to loop mount
an existing root_fs file:
host# mount root_fs uml_root_dir -o loop
You need to change the filesystem type of / in etc/fstab to be
'hostfs', so that line looks like this:
none / hostfs defaults 1 1
Then you need to chown to yourself all the files in that directory
that are owned by root. This worked for me:
host# find . -uid 0 -exec chown jdike {} \;
If you don't want to do that because that's a filesystem image that
you boot as a disk, then run UML as root instead.
Next, make sure that your UML kernel has hostfs compiled in, not as a
module. Then run UML with the following arguments added to the
command line:
root=/dev/root rootflags=/path/to/uml/root rootfstype=hostfs
UML should then boot as it does normally.
[1m9.4. Building hostfs[0m
If you need to build hostfs because it's not in your kernel, you have
two choices:
+o Compiling hostfs into the kernel:
Reconfigure the kernel and set the 'Host filesystem' option under
+o Compiling hostfs as a module:
Reconfigure the kernel and set the 'Host filesystem' option under
be in arch/um/fs/hostfs/hostfs.o. Install that in
/lib/modules/`uname -r`/fs in the virtual machine, boot it up, and
UML# insmod hostfs
[1m10. The Management Console[0m
The UML management console is a low-level interface to the kernel,
somewhat like the i386 SysRq interface. Since there is a full-blown
operating system under UML, there is much greater flexibility possible
than with the SysRq mechanism.
There are a number of things you can do with the mconsole interface:
+o get the kernel version
+o add and remove devices
+o halt or reboot the machine
+o send SysRq commands
+o pause and resume the UML
+o make online backups without shutting down the UML
+o receive notifications of events of interest from within UML
+o monitor the internal state of the UML
You need the mconsole client (uml_mconsole) which is present in CVS
(/tools/mconsole) in 2.4.5-9um and later, and will be in the RPM in
2.4.6.
You also need CONFIG_MCONSOLE (under 'General Setup') enabled in UML.
When you boot UML, you'll see a line like:
mconsole initialized on /home/jdike/.uml/umlNJ32yL/mconsole
If you specify a unique machine id one the UML command line, i.e.
umid=debian
you'll see this
mconsole initialized on /home/jdike/.uml/debian/mconsole
That file is the socket that uml_mconsole will use to communicate with
UML. Run it with either the umid or the full path as its argument:
host% uml_mconsole debian
or
host% uml_mconsole /home/jdike/.uml/debian/mconsole
You'll get a prompt, at which you can run one of these commands:
+o version
+o halt
+o reboot
+o config
+o remove
+o sysrq
+o help
+o cad
+o stop
+o go
+o log
+o proc
[1m10.1. version[0m
This takes no arguments. It prints the UML version.
(mconsole) version
OK Linux usermode 2.4.5-9um #1 Wed Jun 20 22:47:08 EDT 2001 i686
There are a couple actual uses for this. It's a simple no-op which
can be used to check that a UML is running. It's also a way of
sending an interrupt to the UML. This is sometimes useful on SMP
hosts, where there's a bug which causes signals to UML to be lost,
often causing it to appear to hang. Sending such a UML the mconsole
version command is a good way to 'wake it up' before networking has
been enabled, as it does not do anything to the function of the UML.
[1m10.2. halt and reboot[0m
These take no arguments. They shut the machine down immediately, with
no syncing of disks and no clean shutdown of userspace. So, they are
pretty close to crashing the machine.
(mconsole) halt
OK
[1m10.3. config[0m
"config" adds a new device to the virtual machine or queries the
configuration of an existing device.
Currently the ubd and network drivers support pulling devices. It
takes one argument, which is the device to add, with the same syntax
as the kernel command line.
(mconsole)
config ubd3=/home/jdike/incoming/roots/root_fs_debian22
OK
(mconsole) config eth1=mcast
OK
Querying the configuration of a device is handy when you don't know
before the boot what host device the UML device will attach to. This
is a problem with attaching consoles and serial lines to host pty or
pts devices. You have no way of knowing how to access them without
parsing the kernel messages. So, the syntax for this is the same as
above, except you don't specify a configuration
(mconsole) config ssl0
OK pty:/dev/ptyp0
(mconsole) config ubd0
OK /home/jdike/roots/cow.debian,/home/jdike/roots/debian_22
This is supported by the console, serial line, and ubd drivers. As
yet, the network drivers don't support this.
[1m10.4. remove[0m
"remove" deletes a device from the system. Its argument is just the
name of the device to be removed. The device must be idle in whatever
sense the driver considers necessary. In the case of the ubd driver,
the removed block device must not be mounted, swapped on, or otherwise
open, and in the case of the network driver, the device must be down.
(mconsole) remove ubd3
OK
(mconsole) remove eth1
OK
[1m10.5. sysrq[0m
This takes one argument, which is a single letter. It calls the
generic kernel's SysRq driver, which does whatever is called for by
that argument. See the SysRq documentation in Documentation/sysrq.txt
in your favorite kernel tree to see what letters are valid and what
they do.
[1m10.6. help[0m
"help" returns a string listing the valid commands and what each one
does.
[1m10.7. cad[0m
This invokes the Ctl-Alt-Del action on init. What exactly this ends
up doing is up to /etc/inittab. Normally, it reboots the machine.
With UML, this is usually not desired, so if a halt would be better,
then find the section of inittab that looks like this
# What to do when CTRL-ALT-DEL is pressed.
ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
and change the command to halt.
[1m10.8. stop[0m
This puts the UML in a loop reading mconsole requests until a 'go'
mconsole command is recieved. This is very useful for making backups
of UML filesystems, as the UML can be stopped, then synced via 'sysrq
s', so that everything is written to the filesystem. You can then copy
the filesystem and then send the UML 'go' via mconsole.
Note that a UML running with more than one CPU will have problems
after you send the 'stop' command, as only one CPU will be held in a
mconsole loop and all others will continue as normal. This is a bug,
and will be fixed.
[1m10.9. go[0m
This resumes a UML after being paused by a 'stop' command. Note that
when the UML has resumed, TCP connections may have timed out and if
the UML is paused for a long period of time, crond might go a little
crazy, running all the jobs it didn't do earlier.
[1m10.10. log[0m
This takes a string as its argument, and will cause the UML to printk
the string so that it ends up in the kernel message log. This is
intended for use in honeypots by allowing the UML-specific stuff in
the kernel log to be replaced with messages that don't expose the
machine as being a UML.
[1m10.11. proc[0m
This takes a filename as its argument. It will return the contents of
the corresponding /proc file inside the UML. Example:
(mconsole) proc uptime
will return the contents of the UML's /proc/uptime.
[1m10.12. Making online backups[0m
It is possible to make a backup of a UML's data without shutting it
down. The idea is to pause it, make it flush out its data, copy the
filesystem to a safe place, and then resume it. This should usually
take seconds, while shutting down and rebooting the UML could take
minutes. The exact procedure is this:
(mconsole) stop
(mconsole) sysrq s
host% # Copy the UML's filesystem someplace safe
(mconsole) go
By causing UML to flush its data out to disk, the 'sysrq s' will cause
the filesystem to be a clean image. Of course, no guarantees are made
for process data which hadn't been written back to the kernel, but the
filesystem itself won't need an fsck if it's booted.
[1m10.13. Event notification[0m
The mconsole interface also provides a mechanism for processes inside
a UML to send messages to an mconsole client on the host. The
procedure is this:
+o Create a unix socket and pass that to UML on the command line as
the mconsole notification socket
mconsole=notify:socket
+o A /proc/mconsole file will be created inside UML
+o Anything that is written to it will be turned into an mconsole
notification which your mconsole client should be listening for on
the notification socket
A common use for this mechanism is to have an rc script inside UML
send a message out that the UML has booted to a certain stage, and
that something on the host which depends on that can proceed.
However, this is a completely general mechanism which can be used
to communicate any information at all to the host.
There is a demo mconsole notification client in the utilities tarball
in mconsole/notify.pl. This is only a demo, and as such, isn't very
useful by itself. It should be customized to fit into whatever
environment you are setting up.
[1m11. Kernel debugging[0m
This page describes kernel debugging with UML running in tt mode (go
here for the
details on skas and tt mode). Kernel debugging in skas mode is
described here .
Since the UML runs as a normal Linux process, it is possible to debug
it with gdb almost like any other process. It is slightly different
because the kernel's threads are already being ptraced for system call
interception, so gdb can't ptrace them. However, a mechanism has been
added to work around that problem.
In order to debug the kernel, you need build it from source. See
``Compiling the kernel and modules'' for information on doing that.
Make sure that you enable CONFIG_DEBUGSYM and CONFIG_PT_PROXY during
the config. These will compile the kernel with -g, and enable the
ptrace proxy so that gdb works with UML, respectively.
[1m11.1. Starting the kernel under gdb[0m
You can have the kernel running under the control of gdb from the
beginning by putting 'debug' on the command line. You will get an
xterm with gdb running inside it. The kernel will send some commands
to gdb which will leave it stopped at the beginning of start_kernel.
At this point, you can get things going with 'next', 'step', or
'cont'.
There is a transcript of a debugging session here , with breakpoints being set in the scheduler and in an
interrupt handler.
[1m11.2. Examining sleeping processes[0m
Not every bug is evident in the currently running process. Sometimes,
processes hang in the kernel when they shouldn't because they've
deadlocked on a semaphore or something similar. In this case, when
you ^C gdb and get a backtrace, you will see the idle thread, which
isn't very relevant.
What you want is the stack of whatever process is sleeping when it
shouldn't be. You need to figure out which process that is, which is
generally fairly easy. Then you need to get its host process id,
which you can do either by looking at ps on the host or at
task.thread.extern_pid in gdb.
Now what you do is this:
+o detach from the current thread
(UML gdb) det
+o attach to the thread you are interested in
(UML gdb) att
+o look at its stack and anything else of interest
(UML gdb) bt
Note that you can't do anything at this point that requires that a
process execute, e.g. calling a function
+o when you're done looking at that process, reattach to the current
thread and continue it
(UML gdb)
att 1
(UML gdb)
c
Here, specifying any pid which is not the process id of a UML thread
will cause gdb to reattach to the current thread. I commonly use 1,
but any other invalid pid would work.
[1m11.3. Running ddd on UML[0m
ddd works on UML, but requires a special kludge. The process goes
like this:
+o Start ddd
host% ddd linux
+o With ps, get the pid of the gdb that ddd started. You can ask the
gdb to tell you, but for some reason that confuses things and
causes a hang.
+o run UML with 'debug=parent gdb-pid=' added to the command line
- it will just sit there after you hit return
+o type 'att 1' to the ddd gdb and you will see something like
0xa013dc51 in __kill ()
(gdb)
+o At this point, type 'c', UML will boot up, and you can use ddd just
as you do on any other process.
[1m11.4. Debugging modules[0m
gdb has support for debugging code which is dynamically loaded into
the process. This support is what is needed to debug kernel modules
under UML.
Using that support is somewhat complicated. You have to tell gdb what
object file you just loaded into UML and where in memory it is. Then,
it can read the symbol table, and figure out where all the symbols are
from the load address that you provided. It gets more interesting
when you load the module again (i.e. after an rmmod). You have to
tell gdb to forget about all its symbols, including the main UML ones
for some reason, then load then all back in again.
There's an easy way and a hard way to do this. The easy way is to use
the umlgdb expect script written by Chandan Kudige. It basically
automates the process for you.
First, you must tell it where your modules are. There is a list in
the script that looks like this:
set MODULE_PATHS {
"fat" "/usr/src/uml/linux-2.4.18/fs/fat/fat.o"
"isofs" "/usr/src/uml/linux-2.4.18/fs/isofs/isofs.o"
"minix" "/usr/src/uml/linux-2.4.18/fs/minix/minix.o"
}
You change that to list the names and paths of the modules that you
are going to debug. Then you run it from the toplevel directory of
your UML pool and it basically tells you what to do:
******** GDB pid is 21903 ********
Start UML as: ./linux debug gdb-pid=21903
GNU gdb 5.0rh-5 Red Hat Linux 7.1
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) b sys_init_module
Breakpoint 1 at 0xa0011923: file module.c, line 349.
(gdb) att 1
After you run UML and it sits there doing nothing, you hit return at
the 'att 1' and continue it:
Attaching to program: /home/jdike/linux/2.4/um/./linux, process 1
0xa00f4221 in __kill ()
(UML gdb) c
Continuing.
At this point, you debug normally. When you insmod something, the
expect magic will kick in and you'll see something like:
*** Module hostfs loaded ***
Breakpoint 1, sys_init_module (name_user=0x805abb0 "hostfs",
mod_user=0x8070e00) at module.c:349
349 char *name, *n_name, *name_tmp = NULL;
(UML gdb) finish
Run till exit from #0 sys_init_module (name_user=0x805abb0 "hostfs",
mod_user=0x8070e00) at module.c:349
0xa00e2e23 in execute_syscall (r=0xa8140284) at syscall_kern.c:411
411 else res = EXECUTE_SYSCALL(syscall, regs);
Value returned is $1 = 0
(UML gdb)
p/x (int)module_list + module_list->size_of_struct
$2 = 0xa9021054
(UML gdb) symbol-file ./linux
Load new symbol table from "./linux"? (y or n) y
Reading symbols from ./linux...
done.
(UML gdb)
add-symbol-file /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o 0xa9021054
add symbol table from file "/home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o" at
.text_addr = 0xa9021054
(y or n) y
Reading symbols from /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o...
done.
(UML gdb) p *module_list
$1 = {size_of_struct = 84, next = 0xa0178720, name = 0xa9022de0 "hostfs",
size = 9016, uc = {usecount = {counter = 0}, pad = 0}, flags = 1,
nsyms = 57, ndeps = 0, syms = 0xa9023170, deps = 0x0, refs = 0x0,
init = 0xa90221f0 , cleanup = 0xa902222c ,
ex_table_start = 0x0, ex_table_end = 0x0, persist_start = 0x0,
persist_end = 0x0, can_unload = 0, runsize = 0, kallsyms_start = 0x0,
kallsyms_end = 0x0,
archdata_start = 0x1b855 ,
archdata_end = 0xe5890000 ,
kernel_data = 0xf689c35d }
>> Finished loading symbols for hostfs ...
That's the easy way. It's highly recommended. The umlgdb script is
available in the UML utilities tarball in tools/umlgdb/umlgdb.
The hard way is described below in case you're interested in what's
going on.
Boot the kernel under the debugger and load the module with insmod or
modprobe. With gdb, do:
(UML gdb) p module_list
This is a list of modules that have been loaded into the kernel, with
the most recently loaded module first. Normally, the module you want
is at module_list. If it's not, walk down the next links, looking at
the name fields until find the module you want to debug. Take the
address of that structure, and add module.size_of_struct (which in
2.4.10 kernels is 96 (0x60)) to it. Gdb can make this hard addition
for you :-):
(UML gdb)
printf "%#x\n", (int)module_list module_list->size_of_struct
The offset from the module start occasionally changes (before 2.4.0,
it was module.size_of_struct + 4), so it's a good idea to check the
init and cleanup addresses once in a while, as describe below. Now
do:
(UML gdb)
add-symbol-file /path/to/module/on/host that_address
Tell gdb you really want to do it, and you're in business.
If there's any doubt that you got the offset right, like breakpoints
appear not to work, or they're appearing in the wrong place, you can
check it by looking at the module structure. The init and cleanup
fields should look like:
init = 0x588066b0 , cleanup = 0x588066c0
with no offsets on the symbol names. If the names are right, but they
are offset, then the offset tells you how much you need to add to the
address you gave to add-symbol-file.
When you want to load in a new version of the module, you need to get
gdb to forget about the old one. The only way I've found to do that
is to tell gdb to forget about all symbols that it knows about:
(UML gdb) symbol-file
Then reload the symbols from the kernel binary:
(UML gdb) symbol-file /path/to/kernel
and repeat the process above. You'll also need to re-enable break-
points. They were disabled when you dumped all the symbols because
gdb couldn't figure out where they should go.
[1m11.5. Attaching gdb to the kernel[0m
If you don't have the kernel running under gdb, you can attach gdb to
it later by sending the tracing thread a SIGUSR1. The first line of
the console output identifies its pid:
tracing thread pid = 20093
When you send it the signal:
host% kill -USR1 20093
you will get an xterm with gdb running in it.
If you have the mconsole compiled into UML, then the mconsole client
can be used to start gdb:
(mconsole) (mconsole) config gdb=xterm
will fire up an xterm with gdb running in it.
[1m11.6. Using alternate debuggers[0m
UML has support for attaching to an already running debugger rather
than starting gdb itself. This is present in CVS as of 17 Apr 2001.
I sent it to Alan for inclusion in the ac tree, and it will be in my
2.4.4 release.
This is useful when gdb is a subprocess of some UI, such as emacs or
ddd. It can also be used to run debuggers other than gdb on UML.
Below is an example of using strace as an alternate debugger.
To do this, you need to get the pid of the debugger and pass it in
with the
If you are using gdb under some UI, then tell it to 'att 1', and
you'll find yourself attached to UML.
If you are using something other than gdb as your debugger, then
you'll need to get it to do the equivalent of 'att 1' if it doesn't do
it automatically.
An example of an alternate debugger is strace. You can strace the
actual kernel as follows:
+o Run the following in a shell
host%
sh -c 'echo pid=$$; echo -n hit return; read x; exec strace -p 1 -o strace.out'
+o Run UML with 'debug' and 'gdb-pid=' with the pid printed out
by the previous command
+o Hit return in the shell, and UML will start running, and strace
output will start accumulating in the output file.
Note that this is different from running
host% strace ./linux
That will strace only the main UML thread, the tracing thread, which
doesn't do any of the actual kernel work. It just oversees the vir-
tual machine. In contrast, using strace as described above will show
you the low-level activity of the virtual machine.
[1m12. Kernel debugging examples[0m
[1m12.1. The case of the hung fsck[0m
When booting up the kernel, fsck failed, and dropped me into a shell
to fix things up. I ran fsck -y, which hung:
Setting hostname uml [ OK ]
Checking root filesystem
/dev/fhd0 was not cleanly unmounted, check forced.
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.
/dev/fhd0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
[ FAILED ]
*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell.
Give root password for maintenance
(or type Control-D for normal startup):
[root@uml /root]# fsck -y /dev/fhd0
fsck -y /dev/fhd0
Parallelizing fsck version 1.14 (9-Jan-1999)
e2fsck 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
/dev/fhd0 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780. Ignore error? yes
Inode 19780, i_blocks is 1548, should be 540. Fix? yes
Pass 2: Checking directory structure
Error reading block 49405 (Attempt to read block from filesystem resulted in short read). Ignore error? yes
Directory inode 11858, block 0, offset 0: directory corrupted
Salvage? yes
Missing '.' in directory inode 11858.
Fix? yes
Missing '..' in directory inode 11858.
Fix? yes
The standard drill in this sort of situation is to fire up gdb on the
signal thread, which, in this case, was pid 1935. In another window,
I run gdb and attach pid 1935.
~/linux/2.3.26/um 1016: gdb linux
GNU gdb 4.17.0.11 with Linux support
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) att 1935
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1935
0x100756d9 in __wait4 ()
Let's see what's currently running:
(gdb) p current_task.pid
$1 = 0
It's the idle thread, which means that fsck went to sleep for some
reason and never woke up.
Let's guess that the last process in the process list is fsck:
(gdb) p current_task.prev_task.comm
$13 = "fsck.ext2\000\000\000\000\000\000"
It is, so let's see what it thinks it's up to:
(gdb) p current_task.prev_task.thread
$14 = {extern_pid = 1980, tracing = 0, want_tracing = 0, forking = 0,
kernel_stack_page = 0, signal_stack = 1342627840, syscall = {id = 4, args = {
3, 134973440, 1024, 0, 1024}, have_result = 0, result = 50590720},
request = {op = 2, u = {exec = {ip = 1350467584, sp = 2952789424}, fork = {
regs = {1350467584, 2952789424, 0 }, sigstack = 0,
pid = 0}, switch_to = 0x507e8000, thread = {proc = 0x507e8000,
arg = 0xaffffdb0, flags = 0, new_pid = 0}, input_request = {
op = 1350467584, fd = -1342177872, proc = 0, pid = 0}}}}
The interesting things here are the fact that its .thread.syscall.id
is __NR_write (see the big switch in arch/um/kernel/syscall_kern.c or
the defines in include/asm-um/arch/unistd.h), and that it never
returned. Also, its .request.op is OP_SWITCH (see
arch/um/include/user_util.h). These mean that it went into a write,
and, for some reason, called schedule().
The fact that it never returned from write means that its stack should
be fairly interesting. Its pid is 1980 (.thread.extern_pid). That
process is being ptraced by the signal thread, so it must be detached
before gdb can attach it:
(gdb) call detach(1980)
Program received signal SIGSEGV, Segmentation fault.
The program being debugged stopped while in a function called from GDB.
When the function (detach) is done executing, GDB will silently
stop (instead of continuing to evaluate the expression containing
the function call).
(gdb) call detach(1980)
$15 = 0
The first detach segfaults for some reason, and the second one
succeeds.
Now I detach from the signal thread, attach to the fsck thread, and
look at its stack:
(gdb) det
Detaching from program: /home/dike/linux/2.3.26/um/linux Pid 1935
(gdb) att 1980
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1980
0x10070451 in __kill ()
(gdb) bt
#0 0x10070451 in __kill ()
#1 0x10068ccd in usr1_pid (pid=1980) at process.c:30
#2 0x1006a03f in _switch_to (prev=0x50072000, next=0x507e8000)
at process_kern.c:156
#3 0x1006a052 in switch_to (prev=0x50072000, next=0x507e8000, last=0x50072000)
at process_kern.c:161
#4 0x10001d12 in schedule () at sched.c:777
#5 0x1006a744 in __down (sem=0x507d241c) at semaphore.c:71
#6 0x1006aa10 in __down_failed () at semaphore.c:157
#7 0x1006c5d8 in segv_handler (sc=0x5006e940) at trap_user.c:174
#8 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
#9
#10 0x10155404 in errno ()
#11 0x1006c0aa in segv (address=1342179328, is_write=2) at trap_kern.c:50
#12 0x1006c5d8 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
#13 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
#14
#15 0xc0fd in ?? ()
#16 0x10016647 in sys_write (fd=3,
buf=0x80b8800 , count=1024)
at read_write.c:159
#17 0x1006d5b3 in execute_syscall (syscall=4, args=0x5006ef08)
at syscall_kern.c:254
#18 0x1006af87 in really_do_syscall (sig=12) at syscall_user.c:35
#19
#20 0x400dc8b0 in ?? ()
The interesting things here are :
+o There are two segfaults on this stack (frames 9 and 14)
+o The first faulting address (frame 11) is 0x50000800
(gdb) p (void *)1342179328
$16 = (void *) 0x50000800
The initial faulting address is interesting because it is on the idle
thread's stack. I had been seeing the idle thread segfault for no
apparent reason, and the cause looked like stack corruption. In hopes
of catching the culprit in the act, I had turned off all protections
to that stack while the idle thread wasn't running. This apparently
tripped that trap.
However, the more immediate problem is that second segfault and I'm
going to concentrate on that. First, I want to see where the fault
happened, so I have to go look at the sigcontent struct in frame 8:
(gdb) up
#1 0x10068ccd in usr1_pid (pid=1980) at process.c:30
30 kill(pid, SIGUSR1);
(gdb)
#2 0x1006a03f in _switch_to (prev=0x50072000, next=0x507e8000)
at process_kern.c:156
156 usr1_pid(getpid());
(gdb)
#3 0x1006a052 in switch_to (prev=0x50072000, next=0x507e8000, last=0x50072000)
at process_kern.c:161
161 _switch_to(prev, next);
(gdb)
#4 0x10001d12 in schedule () at sched.c:777
777 switch_to(prev, next, prev);
(gdb)
#5 0x1006a744 in __down (sem=0x507d241c) at semaphore.c:71
71 schedule();
(gdb)
#6 0x1006aa10 in __down_failed () at semaphore.c:157
157 }
(gdb)
#7 0x1006c5d8 in segv_handler (sc=0x5006e940) at trap_user.c:174
174 segv(sc->cr2, sc->err & 2);
(gdb)
#8 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
182 segv_handler(sc);
(gdb) p *sc
Cannot access memory at address 0x0.
That's not very useful, so I'll try a more manual method:
(gdb) p *((struct sigcontext *) (&sig + 1))
$19 = {gs = 0, __gsh = 0, fs = 0, __fsh = 0, es = 43, __esh = 0, ds = 43,
__dsh = 0, edi = 1342179328, esi = 1350378548, ebp = 1342630440,
esp = 1342630420, ebx = 1348150624, edx = 1280, ecx = 0, eax = 0,
trapno = 14, err = 4, eip = 268480945, cs = 35, __csh = 0, eflags = 66118,
esp_at_signal = 1342630420, ss = 43, __ssh = 0, fpstate = 0x0, oldmask = 0,
cr2 = 1280}
The ip is in handle_mm_fault:
(gdb) p (void *)268480945
$20 = (void *) 0x1000b1b1
(gdb) i sym $20
handle_mm_fault + 57 in section .text
Specifically, it's in pte_alloc:
(gdb) i line *$20
Line 124 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1b1
and ends at 0x1000b1b7 .
To find where in handle_mm_fault this is, I'll jump forward in the
code until I see an address in that procedure:
(gdb) i line *0x1000b1c0
Line 126 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1b7
and ends at 0x1000b1c3 .
(gdb) i line *0x1000b1d0
Line 131 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1d0
and ends at 0x1000b1da .
(gdb) i line *0x1000b1e0
Line 61 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1da
and ends at 0x1000b1e1 .
(gdb) i line *0x1000b1f0
Line 134 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b1f0
and ends at 0x1000b200 .
(gdb) i line *0x1000b200
Line 135 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b200
and ends at 0x1000b208 .
(gdb) i line *0x1000b210
Line 139 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
starts at address 0x1000b210
and ends at 0x1000b219 .
(gdb) i line *0x1000b220
Line 1168 of "memory.c" starts at address 0x1000b21e
and ends at 0x1000b222 .
Something is apparently wrong with the page tables or vma_structs, so
lets go back to frame 11 and have a look at them:
#11 0x1006c0aa in segv (address=1342179328, is_write=2) at trap_kern.c:50
50 handle_mm_fault(current, vma, address, is_write);
(gdb) call pgd_offset_proc(vma->vm_mm, address)
$22 = (pgd_t *) 0x80a548c
That's pretty bogus. Page tables aren't supposed to be in process
text or data areas. Let's see what's in the vma:
(gdb) p *vma
$23 = {vm_mm = 0x507d2434, vm_start = 0, vm_end = 134512640,
vm_next = 0x80a4f8c, vm_page_prot = {pgprot = 0}, vm_flags = 31200,
vm_avl_height = 2058, vm_avl_left = 0x80a8c94, vm_avl_right = 0x80d1000,
vm_next_share = 0xaffffdb0, vm_pprev_share = 0xaffffe63,
vm_ops = 0xaffffe7a, vm_pgoff = 2952789626, vm_file = 0xafffffec,
vm_private_data = 0x62}
(gdb) p *vma.vm_mm
$24 = {mmap = 0x507d2434, mmap_avl = 0x0, mmap_cache = 0x8048000,
pgd = 0x80a4f8c, mm_users = {counter = 0}, mm_count = {counter = 134904288},
map_count = 134909076, mmap_sem = {count = {counter = 135073792},
sleepers = -1342177872, wait = {lock = ,
task_list = {next = 0xaffffe63, prev = 0xaffffe7a},
__magic = -1342177670, __creator = -1342177300}, __magic = 98},
page_table_lock = {}, context = 138, start_code = 0, end_code = 0,
start_data = 0, end_data = 0, start_brk = 0, brk = 0, start_stack = 0,
arg_start = 0, arg_end = 0, env_start = 0, env_end = 0, rss = 1350381536,
total_vm = 0, locked_vm = 0, def_flags = 0, cpu_vm_mask = 0, swap_cnt = 0,
swap_address = 0, segments = 0x0}
This also pretty bogus. With all of the 0x80xxxxx and 0xaffffxxx
addresses, this is looking like a stack was plonked down on top of
these structures. Maybe it's a stack overflow from the next page:
(gdb) p vma
$25 = (struct vm_area_struct *) 0x507d2434
That's towards the lower quarter of the page, so that would have to
have been pretty heavy stack overflow:
(gdb) x/100x $25
0x507d2434: 0x507d2434 0x00000000 0x08048000 0x080a4f8c
0x507d2444: 0x00000000 0x080a79e0 0x080a8c94 0x080d1000
0x507d2454: 0xaffffdb0 0xaffffe63 0xaffffe7a 0xaffffe7a
0x507d2464: 0xafffffec 0x00000062 0x0000008a 0x00000000
0x507d2474: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2484: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2494: 0x00000000 0x00000000 0x507d2fe0 0x00000000
0x507d24a4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d24b4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d24c4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d24d4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d24e4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d24f4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2504: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2514: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2524: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2534: 0x00000000 0x00000000 0x507d25dc 0x00000000
0x507d2544: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2554: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2564: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2574: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2584: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d2594: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d25a4: 0x00000000 0x00000000 0x00000000 0x00000000
0x507d25b4: 0x00000000 0x00000000 0x00000000 0x00000000
It's not stack overflow. The only "stack-like" piece of this data is
the vma_struct itself.
At this point, I don't see any avenues to pursue, so I just have to
admit that I have no idea what's going on. What I will do, though, is
stick a trap on the segfault handler which will stop if it sees any
writes to the idle thread's stack. That was the thing that happened
first, and it may be that if I can catch it immediately, what's going
on will be somewhat clearer.
[1m12.2. Episode 2: The case of the hung fsck[0m
After setting a trap in the SEGV handler for accesses to the signal
thread's stack, I reran the kernel.
fsck hung again, this time by hitting the trap:
Setting hostname uml [ OK ]
Checking root filesystem
/dev/fhd0 contains a file system with errors, check forced.
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.
/dev/fhd0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
[ FAILED ]
*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell.
Give root password for maintenance
(or type Control-D for normal startup):
[root@uml /root]# fsck -y /dev/fhd0
fsck -y /dev/fhd0
Parallelizing fsck version 1.14 (9-Jan-1999)
e2fsck 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
/dev/fhd0 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780. Ignore error? yes
Pass 2: Checking directory structure
Error reading block 49405 (Attempt to read block from filesystem resulted in short read). Ignore error? yes
Directory inode 11858, block 0, offset 0: directory corrupted
Salvage? yes
Missing '.' in directory inode 11858.
Fix? yes
Missing '..' in directory inode 11858.
Fix? yes
Untested (4127) [100fe44c]: trap_kern.c line 31
I need to get the signal thread to detach from pid 4127 so that I can
attach to it with gdb. This is done by sending it a SIGUSR1, which is
caught by the signal thread, which detaches the process:
kill -USR1 4127
Now I can run gdb on it:
~/linux/2.3.26/um 1034: gdb linux
GNU gdb 4.17.0.11 with Linux support
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) att 4127
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 4127
0x10075891 in __libc_nanosleep ()
The backtrace shows that it was in a write and that the fault address
(address in frame 3) is 0x50000800, which is right in the middle of
the signal thread's stack page:
(gdb) bt
#0 0x10075891 in __libc_nanosleep ()
#1 0x1007584d in __sleep (seconds=1000000)
at ../sysdeps/unix/sysv/linux/sleep.c:78
#2 0x1006ce9a in stop () at user_util.c:191
#3 0x1006bf88 in segv (address=1342179328, is_write=2) at trap_kern.c:31
#4 0x1006c628 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
#5 0x1006c63c in kern_segv_handler (sig=11) at trap_user.c:182
#6
#7 0xc0fd in ?? ()
#8 0x10016647 in sys_write (fd=3, buf=0x80b8800 "R.", count=1024)
at read_write.c:159
#9 0x1006d603 in execute_syscall (syscall=4, args=0x5006ef08)
at syscall_kern.c:254
#10 0x1006af87 in really_do_syscall (sig=12) at syscall_user.c:35
#11
#12 0x400dc8b0 in ?? ()
#13
#14 0x400dc8b0 in ?? ()
#15 0x80545fd in ?? ()
#16 0x804daae in ?? ()
#17 0x8054334 in ?? ()
#18 0x804d23e in ?? ()
#19 0x8049632 in ?? ()
#20 0x80491d2 in ?? ()
#21 0x80596b5 in ?? ()
(gdb) p (void *)1342179328
$3 = (void *) 0x50000800
Going up the stack to the segv_handler frame and looking at where in
the code the access happened shows that it happened near line 110 of
block_dev.c:
(gdb) up
#1 0x1007584d in __sleep (seconds=1000000)
at ../sysdeps/unix/sysv/linux/sleep.c:78
../sysdeps/unix/sysv/linux/sleep.c:78: No such file or directory.
(gdb)
#2 0x1006ce9a in stop () at user_util.c:191
191 while(1) sleep(1000000);
(gdb)
#3 0x1006bf88 in segv (address=1342179328, is_write=2) at trap_kern.c:31
31 KERN_UNTESTED();
(gdb)
#4 0x1006c628 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
174 segv(sc->cr2, sc->err & 2);
(gdb) p *sc
$1 = {gs = 0, __gsh = 0, fs = 0, __fsh = 0, es = 43, __esh = 0, ds = 43,
__dsh = 0, edi = 1342179328, esi = 134973440, ebp = 1342631484,
esp = 1342630864, ebx = 256, edx = 0, ecx = 256, eax = 1024, trapno = 14,
err = 6, eip = 268550834, cs = 35, __csh = 0, eflags = 66070,
esp_at_signal = 1342630864, ss = 43, __ssh = 0, fpstate = 0x0, oldmask = 0,
cr2 = 1342179328}
(gdb) p (void *)268550834
$2 = (void *) 0x1001c2b2
(gdb) i sym $2
block_write + 1090 in section .text
(gdb) i line *$2
Line 209 of "/home/dike/linux/2.3.26/um/include/asm/arch/string.h"
starts at address 0x1001c2a1
and ends at 0x1001c2bf .
(gdb) i line *0x1001c2c0
Line 110 of "block_dev.c" starts at address 0x1001c2bf
and ends at 0x1001c2e3 .
Looking at the source shows that the fault happened during a call to
copy_to_user to copy the data into the kernel:
107 count -= chars;
108 copy_from_user(p,buf,chars);
109 p += chars;
110 buf += chars;
p is the pointer which must contain 0x50000800, since buf contains
0x80b8800 (frame 8 above). It is defined as:
p = offset + bh->b_data;
I need to figure out what bh is, and it just so happens that bh is
passed as an argument to mark_buffer_uptodate and mark_buffer_dirty a
few lines later, so I do a little disassembly:
(gdb) disas 0x1001c2bf 0x1001c2e0
Dump of assembler code from 0x1001c2bf to 0x1001c2d0:
0x1001c2bf : addl %eax,0xc(%ebp)
0x1001c2c2 : movl 0xfffffdd4(%ebp),%edx
0x1001c2c8 : btsl $0x0,0x18(%edx)
0x1001c2cd : btsl $0x1,0x18(%edx)
0x1001c2d2 : sbbl %ecx,%ecx
0x1001c2d4 : testl %ecx,%ecx
0x1001c2d6 : jne 0x1001c2e3
0x1001c2d8 : pushl $0x0
0x1001c2da : pushl %edx
0x1001c2db : call 0x1001819c <__mark_buffer_dirty>
End of assembler dump.
At that point, bh is in %edx (address 0x1001c2da), which is calculated
at 0x1001c2c2 as %ebp + 0xfffffdd4, so I figure exactly what that is,
taking %ebp from the sigcontext_struct above:
(gdb) p (void *)1342631484
$5 = (void *) 0x5006ee3c
(gdb) p 0x5006ee3c+0xfffffdd4
$6 = 1342630928
(gdb) p (void *)$6
$7 = (void *) 0x5006ec10
(gdb) p *((void **)$7)
$8 = (void *) 0x50100200
Now, I look at the structure to see what's in it, and particularly,
what its b_data field contains:
(gdb) p *((struct buffer_head *)0x50100200)
$13 = {b_next = 0x50289380, b_blocknr = 49405, b_size = 1024, b_list = 0,
b_dev = 15872, b_count = {counter = 1}, b_rdev = 15872, b_state = 24,
b_flushtime = 0, b_next_free = 0x501001a0, b_prev_free = 0x50100260,
b_this_page = 0x501001a0, b_reqnext = 0x0, b_pprev = 0x507fcf58,
b_data = 0x50000800 "", b_page = 0x50004000,
b_end_io = 0x10017f60 , b_dev_id = 0x0,
b_rsector = 98810, b_wait = {lock = ,
task_list = {next = 0x50100248, prev = 0x50100248}, __magic = 1343226448,
__creator = 0}, b_kiobuf = 0x0}
The b_data field is indeed 0x50000800, so the question becomes how
that happened. The rest of the structure looks fine, so this probably
is not a case of data corruption. It happened on purpose somehow.
The b_page field is a pointer to the page_struct representing the
0x50000000 page. Looking at it shows the kernel's idea of the state
of that page:
(gdb) p *$13.b_page
$17 = {list = {next = 0x50004a5c, prev = 0x100c5174}, mapping = 0x0,
index = 0, next_hash = 0x0, count = {counter = 1}, flags = 132, lru = {
next = 0x50008460, prev = 0x50019350}, wait = {
lock = , task_list = {next = 0x50004024,
prev = 0x50004024}, __magic = 1342193708, __creator = 0},
pprev_hash = 0x0, buffers = 0x501002c0, virtual = 1342177280,
zone = 0x100c5160}
Some sanity-checking: the virtual field shows the "virtual" address of
this page, which in this kernel is the same as its "physical" address,
and the page_struct itself should be mem_map[0], since it represents
the first page of memory:
(gdb) p (void *)1342177280
$18 = (void *) 0x50000000
(gdb) p mem_map
$19 = (mem_map_t *) 0x50004000
These check out fine.
Now to check out the page_struct itself. In particular, the flags
field shows whether the page is considered free or not:
(gdb) p (void *)132
$21 = (void *) 0x84
The "reserved" bit is the high bit, which is definitely not set, so
the kernel considers the signal stack page to be free and available to
be used.
At this point, I jump to conclusions and start looking at my early
boot code, because that's where that page is supposed to be reserved.
In my setup_arch procedure, I have the following code which looks just
fine:
bootmap_size = init_bootmem(start_pfn, end_pfn - start_pfn);
free_bootmem(__pa(low_physmem) + bootmap_size, high_physmem - low_physmem);
Two stack pages have already been allocated, and low_physmem points to
the third page, which is the beginning of free memory.
The init_bootmem call declares the entire memory to the boot memory
manager, which marks it all reserved. The free_bootmem call frees up
all of it, except for the first two pages. This looks correct to me.
So, I decide to see init_bootmem run and make sure that it is marking
those first two pages as reserved. I never get that far.
Stepping into init_bootmem, and looking at bootmem_map before looking
at what it contains shows the following:
(gdb) p bootmem_map
$3 = (void *) 0x50000000
Aha! The light dawns. That first page is doing double duty as a
stack and as the boot memory map. The last thing that the boot memory
manager does is to free the pages used by its memory map, so this page
is getting freed even its marked as reserved.
The fix was to initialize the boot memory manager before allocating
those two stack pages, and then allocate them through the boot memory
manager. After doing this, and fixing a couple of subsequent buglets,
the stack corruption problem disappeared.
[1m13. What to do when UML doesn't work[0m
[1m13.1. Strange compilation errors when you build from source[0m
As of test11, it is necessary to have "ARCH=um" in the environment or
on the make command line for all steps in building UML, including
clean, distclean, or mrproper, config, menuconfig, or xconfig, dep,
and linux. If you forget for any of them, the i386 build seems to
contaminate the UML build. If this happens, start from scratch with
host%
make mrproper ARCH=um
and repeat the build process with ARCH=um on all the steps.
See ``Compiling the kernel and modules'' for more details.
Another cause of strange compilation errors is building UML in
/usr/src/linux. If you do this, the first thing you need to do is
clean up the mess you made. The /usr/src/linux/asm link will now
point to /usr/src/linux/asm-um. Make it point back to
/usr/src/linux/asm-i386. Then, move your UML pool someplace else and
build it there. Also see below, where a more specific set of symptoms
is described.
[1m13.2. UML hangs on boot after mounting devfs[0m
The boot looks like this:
VFS: Mounted root (ext2 filesystem) readonly.
Mounted devfs on /dev
You're probably running a recent distribution on an old machine. I
saw this with the RH7.1 filesystem running on a Pentium. The shared
library loader, ld.so, was executing an instruction (cmove) which the
Pentium didn't support. That instruction was apparently added later.
If you run UML under the debugger, you'll see the hang caused by one
instruction causing an infinite SIGILL stream.
The fix is to boot UML on an older filesystem.
[1m13.3. A variety of panics and hangs with /tmp on a reiserfs filesys-[0m
[1mtem[0m
I saw this on reiserfs 3.5.21 and it seems to be fixed in 3.5.27.
Panics preceded by
Detaching pid nnnn
are diagnostic of this problem. This is a reiserfs bug which causes a
thread to occasionally read stale data from a mmapped page shared with
another thread. The fix is to upgrade the filesystem or to have /tmp
be an ext2 filesystem.
[1m13.4. The compile fails with errors about conflicting types for[0m
[1m'open', 'dup', and 'waitpid'[0m
This happens when you build in /usr/src/linux. The UML build makes
the include/asm link point to include/asm-um. /usr/include/asm points
to /usr/src/linux/include/asm, so when that link gets moved, files
which need to include the asm-i386 versions of headers get the
incompatible asm-um versions. The fix is to move the include/asm link
back to include/asm-i386 and to do UML builds someplace else.
[1m13.5. UML doesn't work when /tmp is an NFS filesystem[0m
This seems to be a similar situation with the resierfs problem above.
Some versions of NFS seems not to handle mmap correctly, which UML
depends on. The workaround is have /tmp be non-NFS directory.
[1m13.6. UML hangs on boot when compiled with gprof support[0m
If you build UML with gprof support and, early in the boot, it does
this
kernel BUG at page_alloc.c:100!
you have a buggy gcc. You can work around the problem by removing
UM_FASTCALL from CFLAGS in arch/um/Makefile-i386. This will open up
another bug, but that one is fairly hard to reproduce.
[1m13.7. syslogd dies with a SIGTERM on startup[0m
The exact boot error depends on the distribution that you're booting,
but Debian produces this:
/etc/rc2.d/S10sysklogd: line 49: 93 Terminated
start-stop-daemon --start --quiet --exec /sbin/syslogd -- $SYSLOGD
This is a syslogd bug. There's a race between a parent process
installing a signal handler and its child sending the signal. See
this uml-devel post for the details.
[1m13.8. TUN/TAP networking doesn't work on a 2.4 host[0m
There are a couple of problems which were
name="pointed
out"> by Tim Robinson
+o It doesn't work on hosts running 2.4.7 (or thereabouts) or earlier.
The fix is to upgrade to something more recent and then read the
next item.
+o If you see
File descriptor in bad state
when you bring up the device inside UML, you have a header mismatch
between the original kernel and the upgraded one. Make /usr/src/linux
point at the new headers. This will only be a problem if you build
uml_net yourself.
[1m13.9. You can network to the host but not to other machines on the[0m
[1mnet[0m
If you can connect to the host, and the host can connect to UML, but
you can not connect to any other machines, then you may need to enable
IP Masquerading on the host. Usually this is only experienced when
using private IP addresses (192.168.x.x or 10.x.x.x) for host/UML
networking, rather than the public address space that your host is
connected to. UML does not enable IP Masquerading, so you will need
to create a static rule to enable it:
host%
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Replace eth0 with the interface that you use to talk to the rest of
the world.
Documentation on IP Masquerading, and SNAT, can be found at
www.netfilter.org .
If you can reach the local net, but not the outside Internet, then
that is usually a routing problem. The UML needs a default route:
UML#
route add default gw gateway IP
The gateway IP can be any machine on the local net that knows how to
reach the outside world. Usually, this is the host or the local net-
work's gateway.
Occasionally, we hear from someone who can reach some machines, but
not others on the same net, or who can reach some ports on other
machines, but not others. These are usually caused by strange
firewalling somewhere between the UML and the other box. You track
this down by running tcpdump on every interface the packets travel
over and see where they disappear. When you find a machine that takes
the packets in, but does not send them onward, that's the culprit.
[1m13.10. I have no root and I want to scream[0m
Thanks to Birgit Wahlich for telling me about this strange one. It
turns out that there's a limit of six environment variables on the
kernel command line. When that limit is reached or exceeded, argument
processing stops, which means that the 'root=' argument that UML
usually adds is not seen. So, the filesystem has no idea what the
root device is, so it panics.
The fix is to put less stuff on the command line. Glomming all your
setup variables into one is probably the best way to go.
[1m13.11. UML build conflict between ptrace.h and ucontext.h[0m
On some older systems, /usr/include/asm/ptrace.h and
/usr/include/sys/ucontext.h define the same names. So, when they're
included together, the defines from one completely mess up the parsing
of the other, producing errors like:
/usr/include/sys/ucontext.h:47: parse error before
`10'
plus a pile of warnings.
This is a libc botch, which has since been fixed, and I don't see any
way around it besides upgrading.
[1m13.12. The UML BogoMips is exactly half the host's BogoMips[0m
On i386 kernels, there are two ways of running the loop that is used
to calculate the BogoMips rating, using the TSC if it's there or using
a one-instruction loop. The TSC produces twice the BogoMips as the
loop. UML uses the loop, since it has nothing resembling a TSC, and
will get almost exactly the same BogoMips as a host using the loop.
However, on a host with a TSC, its BogoMips will be double the loop
BogoMips, and therefore double the UML BogoMips.
[1m13.13. When you run UML, it immediately segfaults[0m
If the host is configured with the 2G/2G address space split, that's
why. See ``UML on 2G/2G hosts'' for the details on getting UML to
run on your host.
[1m13.14. xterms appear, then immediately disappear[0m
If you're running an up to date kernel with an old release of
uml_utilities, the port-helper program will not work properly, so
xterms will exit straight after they appear. The solution is to
upgrade to the latest release of uml_utilities. Usually this problem
occurs when you have installed a packaged release of UML then compiled
your own development kernel without upgrading the uml_utilities from
the source distribution.
[1m13.15. cannot set up thread-local storage[0m
This problem is fixed by the skas-hold-own-ldt patch that went into
2.6.15-rc1.
The boot looks like this:
cannot set up thread-local storage: cannot set up LDT for thread-local storage
Kernel panic - not syncing: Attempted to kill init!
Your UML kernel doesn't support Native Posix Thread Library (NPTL) and
the binaries you're running are being dynamically linked to it. Try
running in SKAS3 mode first. You might be able to avoid the kernel
panic setting the LD_ASSUME_KERNEL
environment
variable on the command line:
./linux init=/bin/sh LD_ASSUME_KERNEL=2.4.1
Many commands are very restrictive about what is preserved in the
environment when starting child processes, so relying on
LD_ASSUME_KERNEL to be globally set for all processes in the whole
system is generally not a good idea. It's very hard to guarantee.
Thus it's better to move the NPTL libraries away:
# mount root_fs mnt-uml/ -o loop
# mv mnt-uml/lib/tls mnt-uml/lib/tls.away
# umount mnt-uml
If you're running Debian, you might prefer to use dpkg-divert:
# export LD_ASSUME_KERNEL=2.4.1
# mount root_fs mnt-uml/ -o loop
# chroot mnt-uml
# mkdir /lib/tls.off
# cd /lib/tls
# for f in *;
do
dpkg-divert --divert --local --rename --divert /lib/tls.off/$f --add /lib/tls/$f;
done
# exit
# umount mnt-uml
# unset LD_ASSUME_KERNEL
[1m13.16. Process segfaults with a modern (NPTL-using) filesystem[0m
These appear to be fixed with the tls patches from Blaisorblade that
are currently in my patchset . You can apply the entire
patchset, or you can move /lib/tls in the image away, as described
above.
[1m13.17. Any other panic, hang, or strange behavior[0m
If you're seeing truly strange behavior, such as hangs or panics that
happen in random places, or you try running the debugger to see what's
happening and it acts strangely, then it could be a problem in the
host kernel. If you're not running a stock Linus or -ac kernel, then
try that. An early version of the preemption patch and a 2.4.10 SuSE
kernel have caused very strange problems in UML.
Otherwise, let me know about it. Send a message to one of the UML
mailing lists - either the developer list - user-mode-linux-devel at
lists dot sourceforge dot net (subscription info) or the user list -
user-mode-linux-user at lists dot sourceforge do net (subscription
info), whichever you prefer. Don't assume that everyone knows about
it and that a fix is imminent.
If you want to be super-helpful, read ``Diagnosing Problems'' and
follow the instructions contained therein.
[1m14. Diagnosing Problems[0m
If you get UML to crash, hang, or otherwise misbehave, you should
report this on one of the project mailing lists, either the developer
list - user-mode-linux-devel at lists dot sourceforge dot net
(subscription info) or the user list - user-mode-linux-user at lists
dot sourceforge dot net (subscription info). When you do, it is
likely that I will want more information. So, it would be helpful to
read the stuff below, do whatever is applicable in your case, and
report the results to the list.
For any diagnosis, you're going to need to build a debugging kernel.
The binaries from this site aren't debuggable. If you haven't done
this before, read about ``Compiling the kernel and modules'' and
``Kernel debugging'' UML first.
[1m14.1. Case 1 : Normal kernel panics[0m
The most common case is for a normal thread to panic. To debug this,
you will need to run it under the debugger (add 'debug' to the command
line). An xterm will start up with gdb running inside it. Continue
it when it stops in start_kernel and make it crash. Now ^C gdb and
If the panic was a "Kernel mode fault", then there will be a segv
frame on the stack and I'm going to want some more information. The
stack might look something like this:
(UML gdb) backtrace
#0 0x1009bf76 in __sigprocmask (how=1, set=0x5f347940, oset=0x0)
at ../sysdeps/unix/sysv/linux/sigprocmask.c:49
#1 0x10091411 in change_sig (signal=10, on=1) at process.c:218
#2 0x10094785 in timer_handler (sig=26) at time_kern.c:32
#3 0x1009bf38 in __restore ()
at ../sysdeps/unix/sysv/linux/i386/sigaction.c:125
#4 0x1009534c in segv (address=8, ip=268849158, is_write=2, is_user=0)
at trap_kern.c:66
#5 0x10095c04 in segv_handler (sig=11) at trap_user.c:285
#6 0x1009bf38 in __restore ()
I'm going to want to see the symbol and line information for the value
of ip in the segv frame. In this case, you would do the following:
(UML gdb) i sym 268849158
and
(UML gdb) i line *268849158
The reason for this is the __restore frame right above the segv_han-
dler frame is hiding the frame that actually segfaulted. So, I have
to get that information from the faulting ip.
[1m14.2. Case 2 : Tracing thread panics[0m
The less common and more painful case is when the tracing thread
panics. In this case, the kernel debugger will be useless because it
needs a healthy tracing thread in order to work. The first thing to
do is get a backtrace from the tracing thread. This is done by
figuring out what its pid is, firing up gdb, and attaching it to that
pid. You can figure out the tracing thread pid by looking at the
first line of the console output, which will look like this:
tracing thread pid = 15851
or by running ps on the host and finding the line that looks like
this:
jdike 15851 4.5 0.4 132568 1104 pts/0 S 21:34 0:05 ./linux [(tracing thread)]
If the panic was 'segfault in signals', then follow the instructions
above for collecting information about the location of the seg fault.
If the tracing thread flaked out all by itself, then send that
backtrace in and wait for our crack debugging team to fix the problem.
[1m14.3. Case 3 : Tracing thread panics caused by other threads[0m
However, there are cases where the misbehavior of another thread
caused the problem. The most common panic of this type is:
wait_for_stop failed to wait for to stop with
In this case, you'll need to get a backtrace from the process men-
tioned in the panic, which is complicated by the fact that the kernel
debugger is defunct and without some fancy footwork, another gdb can't
attach to it. So, this is how the fancy footwork goes:
In a shell:
host% kill -STOP pid
Run gdb on the tracing thread as described in case 2 and do:
(host gdb) call detach(pid)
If you get a segfault, do it again. It always works the second time.
Detach from the tracing thread and attach to that other thread:
(host gdb) detach
(host gdb) attach pid
If gdb hangs when attaching to that process, go back to a shell and
do:
host%
kill -CONT pid
And then get the backtrace:
(host gdb) backtrace
[1m14.4. Case 4 : Hangs[0m
Hangs seem to be fairly rare, but they sometimes happen. When a hang
happens, we need a backtrace from the offending process. Run the
kernel debugger as described in case 1 and get a backtrace. If the
current process is not the idle thread, then send in the backtrace.
You can tell that it's the idle thread if the stack looks like this:
#0 0x100b1401 in __libc_nanosleep ()
#1 0x100a2885 in idle_sleep (secs=10) at time.c:122
#2 0x100a546f in do_idle () at process_kern.c:445
#3 0x100a5508 in cpu_idle () at process_kern.c:471
#4 0x100ec18f in start_kernel () at init/main.c:592
#5 0x100a3e10 in start_kernel_proc (unused=0x0) at um_arch.c:71
#6 0x100a383f in signal_tramp (arg=0x100a3dd8) at trap_user.c:50
If this is the case, then some other process is at fault, and went to
sleep when it shouldn't have. Run ps on the host and figure out which
process should not have gone to sleep and stayed asleep. Then attach
to it with gdb and get a backtrace as described in case 3.
[1m15. Thanks[0m
A number of people have helped this project in various ways, and this
page gives recognition where recognition is due.
If you're listed here and you would prefer a real link on your name,
or no link at all, instead of the despammed email address pseudo-link,
let me know.
If you're not listed here and you think maybe you should be, please
let me know that as well. I try to get everyone, but sometimes my
bookkeeping lapses and I forget about contributions.
[1m15.1. Code and Documentation[0m
Rusty Russell -
+o wrote the HOWTO
+o prodded me into making this project official and putting it on
SourceForge
+o came up with the way cool UML logo
+o redid the config process
Peter Moulder - Fixed my config and build
processes, and added some useful code to the block driver
Bill Stearns -
+o HOWTO updates
+o lots of bug reports
+o lots of testing
+o dedicated a box (uml.ists.dartmouth.edu) to support UML development
+o wrote the mkrootfs script, which allows bootable filesystems of
RPM-based distributions to be cranked out
+o cranked out a large number of filesystems with said script
Jim Leu - Wrote the virtual ethernet driver
and associated usermode tools
Lars Brinkhoff - Contributed the ptrace
proxy from his own project to allow easier
kernel debugging
Andrea Arcangeli - Redid some of the early boot
code so that it would work on machines with Large File Support
Chris Emerson - Did
the first UML port to Linux/ppc
Harald Welte - Wrote the multicast
transport for the network driver
Jorgen Cederlof - Added special file support to hostfs
Greg Lonnon - Changed the ubd driver
to allow it to layer a COW file on a shared read-only filesystem and
wrote the iomem emulation support
Henrik Nordstrom - Provided a variety
of patches, fixes, and clues
Lennert Buytenhek - Contributed various patches, a rewrite of the
network driver, the first implementation of the mconsole driver, and
did the bulk of the work needed to get SMP working again.
Yon Uriarte - Fixed the TUN/TAP network backend while I slept.
Adam Heath - Made a bunch of nice cleanups to the initialization code,
plus various other small patches.
Matt Zimmerman - Matt volunteered to be the UML Debian maintainer and
is doing a real nice job of it. He also noticed and fixed a number of
actually and potentially exploitable security holes in uml_net. Plus
the occasional patch. I like patches.
James McMechan - James seems to have taken over maintenance of the ubd
driver and is doing a nice job of it.
Chandan Kudige - wrote the umlgdb script which automates the reloading
of module symbols.
Steve Schmidtke - wrote the UML slirp transport and hostaudio drivers,
enabling UML processes to access audio devices on the host. He also
submitted patches for the slip transport and lots of other things.
David Coulson -
+o Set up the usermodelinux.org site,
which is a great way of keeping the UML user community on top of
UML goings-on.
+o Site documentation and updates
+o Nifty little UML management daemon UMLd
+o Lots of testing and bug reports
[1m15.2. Flushing out bugs[0m
+o Yuri Pudgorodsky
+o Gerald Britton
+o Ian Wehrman
+o Gord Lamb
+o Eugene Koontz
+o John H. Hartman
+o Anders Karlsson
+o Daniel Phillips
+o John Fremlin
+o Rainer Burgstaller
+o James Stevenson
+o Matt Clay
+o Cliff Jefferies
+o Geoff Hoff
+o Lennert Buytenhek
+o Al Viro
+o Frank Klingenhoefer
+o Livio Baldini Soares
+o Jon Burgess
+o Petru Paler
+o Paul
+o Chris Reahard
+o Sverker Nilsson
+o Gong Su
+o johan verrept
+o Bjorn Eriksson
+o Lorenzo Allegrucci
+o Muli Ben-Yehuda
+o David Mansfield
+o Howard Goff
+o Mike Anderson
+o John Byrne
+o Sapan J. Batia
+o Iris Huang
+o Jan Hudec
+o Voluspa
[1m15.3. Buglets and clean-ups[0m
+o Dave Zarzycki
+o Adam Lazur
+o Boria Feigin
+o Brian J. Murrell
+o JS
+o Roman Zippel
+o Wil Cooley
+o Ayelet Shemesh
+o Will Dyson
+o Sverker Nilsson
+o dvorak
+o v.naga srinivas
+o Shlomi Fish
+o Roger Binns
+o johan verrept
+o MrChuoi
+o Peter Cleve
+o Vincent Guffens
+o Nathan Scott
+o Patrick Caulfield
+o jbearce
+o Catalin Marinas
+o Shane Spencer
+o Zou Min
+o Ryan Boder
+o Lorenzo Colitti
+o Gwendal Grignou
+o Andre' Breiler
+o Tsutomu Yasuda
[1m15.4. Case Studies[0m
+o Jon Wright
+o William McEwan
+o Michael Richardson
[1m15.5. Other contributions[0m
Bill Carr made the Red Hat mkrootfs script
work with RH 6.2.
Michael Jennings sent in some material which
is now gracing the top of the index page of this site.
SGI (and more specifically Ralf Baechle ) gave me an account on oss.sgi.com
. The bandwidth there made it possible to
produce most of the filesystems available on the project download
page.
Laurent Bonnaud took the old grotty
Debian filesystem that I've been distributing and updated it to 2.2.
It is now available by itself here.
Rik van Riel gave me some ftp space on ftp.nl.linux.org so I can make
releases even when Sourceforge is broken.
Rodrigo de Castro looked at my broken pte code and told me what was
wrong with it, letting me fix a long-standing (several weeks) and
serious set of bugs.
Chris Reahard built a specialized root filesystem for running a DNS
server jailed inside UML. It's available from the download
page in the Jail
Filesysems section.
user-mode-linux-doc-20060501/arch-port.html 0000644 0000000 0000000 00000122773 12742461304 015243 0 ustar
Porting UML to a new architecture
Even though UML is running on a host Linux, which insulates it from
the underlying platform to a great extent, some details of the hardware
still leak through and make porting UML to Linux on a new architecture
more than a simple rebuild.
The major aspects of the hardware that show through are
register names used by ptrace
organization of the process address space
This page will describe how to port UML to a new architecture. It
will acquire new material as we learn more about how to do it. At
this point, this is based on what we learned from the ppc port, which
is the only real port of UML that's been done so far. The i386 port
doesn't really count since that was part of the overall development of
UML rather than a separate porting effort.
Below, there are references to $(SUBARCH). This is the make variable
which holds the value of the host architecture in the UML build. On
Intel boxes, it's "i386" and on PowerPC boxes, it's "ppc".
Overview
UML is split between architecture-independent code and headers which
are found under arch/um in
kernel
drivers
fs
ptproxy
include
and the architecture-dependent code and definitions under arch/um in
Makefile-*
sys-*
include/sysdep-*
Each '*' is the name of an architecture, so the i386-specific code is
under arch/um/sys-i386 and the ppc-specific code is under arch/um/sys-ppc.
The host
Not all architectures can currently run UML. The potential problem is
the ability of ptrace to change system call numbers. i386 couldn't
until I got the change into 2.3.22 and 2.2.15, ppc could, and IA64 and
mips can't. I don't know about the other arches.
This is necessary because it's critical to UML's ability to virtualize
system calls. Process system calls must be nullified in the host, and
this is done by converting them into getpid.
So, before starting to work on your new port of UML, make sure ptrace
is fully functional. This
little program starts a child, which makes calls to getpid and
prints them out, while the parent is converting the getpid calls to
getppid. The parent prints out its own pid, while the child prints out
what it thinks is its own pid, and they should be the same. So, if
your machine is able to run UML, you will see output like this:
Parent pid = 3246
getpid() returned 3246
getpid() returned 3246
If not, you will likely get errors from ptrace. Less likely is
different pids being printed out from the two processes. If either
happens, then you need to figure out how to remove that restriction
from the host Linux.
Note that when you compile ptrace.c, you will need to change the
references to ORIG_EAX, which contains the system call number, to
whatever is appropriate for your architecture.
Address space layout
Before delving into the code, you need to do some high-level
conceptual thinking about how to organize the address space of a UML
process. UML maps its executable, physical memory, and kernel virtual
memory into the address space of each of its processes. You need to
decide where to put each of these so as to minimize the likelihood of
a process trying to allocate that memory for its own use.
The only arch hook at this point is where in the address space the UML
binary is going to load. The other addresses are still hard-coded
because they happen to work for both i386 and ppc. UML puts its own
memory in the area starting at 0x5000000 and process stacks 4M below
its own process stack. These choices may not work on all
architectures, so feel free to generalize them. To locate a likely
area on your arch, staring at /proc/<pid>/maps of various processes
on the host has been the technique so far.
Architecture Makefile
You need to create arch/um/Makefile-$(SUBARCH), which contains the
following definitions:
START_ADDR - The address where the UML executable will load in
memory. This address must be chosen so that it won't conflict with
any memory that a UML process is going to want to use. The i386
definition is
START_ADDR = 0x10000000
ARCH_CFLAGS - Anything that needs to be added to CFLAGS goes here.
Both the i386 and ppc ports use this to turn off definitions that
would pull hardware-specific code into the kernel. The ppc definition
is
ARCH_CFLAGS = -U__powerpc__ -D__UM_PPC__
ELF_SUBARCH - This is the name of the ELF object format for the
architecture. On i386, it's 'i386', but on ppc, it's not 'ppc' (it's
'powerpc'). The i386 definition is
ELF_SUBARCH = $(SUBARCH)
include/asm-um
Every architecture needs to provide a set of headers to the generic
kernel. These are located in include/asm-$(ARCH). UML mostly uses
the headers of the underlying architecture. It does this by creating
a symlink from include/asm-um/arch to include/asm-$(SUBARCH). Most of
UML's headers then just include "asm/arch/header.h". As an example,
this is rwlock.h
Almost all of UML's headers look exactly like that. Some of the rest
are architecture-independent headers private to UML. You don't need to
worry about these.
Some headers include the underlying arch headers, but also need to
change them in some way. For example, the UML ptrace-generic.h includes
asm/arch/ptrace.h because it needs some definitions from there.
However, there are things it doesn't want because it needs to define
its own, such as struct pt_regs. So, the underlying architecture's
struct pt_regs is renamed by doing the following
This changes the name of the underlying architecture's pt_regs to
struct pt_regs_subarch, allowing UML to define its own struct
pt_regs. This practice of taking most of the contents other
architecture's header and defining the unwanted bits away is useful,
but it also causes problems which porter have to deal with. For
example, the ppc ptrace.h includes "asm/system.h", which includes the
UML system.h. Since the UML system.h contains references to pt_regs
and it's being included by a header that has had pt_regs renamed to
pt_regs_subarch, the UML system.h references are similarly renamed.
This causes conflicts against files which expect references to
pt_regs. There have been several attempts to update UML/ppc and this
problem has stymied them. It's far from an insolvable problem, but it
involves staring at confusing sequences of includes, figuring out
what's happening, and how to fix it.
There are also a few headers which are archtecture-dependent.
archparam-$(SUBARCH).h
This is a header for miscellaneous architecture-dependent
definitions. In a lot of cases, a mostly-generic header can get its
non-generic definitions from a separate header. In these cases, the
definitions are put in archparam-$(SUBARCH).h and that header includes
asm/archparam.h, which is a symbolic link to archparam-$(SUBARCH).h.
The i386 archparam mostly includes definitions for elf.h, such as
platform-specific register initializatins. The ppc archparam is
similar, adding a couple of definitions for hardirq.h and a couple of
other headers.
processor-$(SUBARCH).h
This header exists because UML/ppc needs a little logic to choose
between including one header and another.
ptrace-$(SUBARCH).h
This defines architecture-dependent access macros into struct
pt_regs. These macros aren't used in generic code, but you may need
some when you define ELF_PLAT_INIT and a few other things. Those
definitions should go here.
sigcontext-$(SUBARCH).h
This is similar to processor-$(SUBARCH).h. It exists so that that the
occurrences of pt_regs in asm-$(SUBARCH)/sigcontext.h can be defined
out of the way.
system-$(SUBARCH).h
This header exists so that UML/ppc can define the ppc _switch_to out
of the way. The i386 version doesn't do anything.
ptrace-generic.h
ptrace-generic.h isn't architecture-dependent, but it does put some
requirements on arch/um/include/sysdep/ptrace.h, which you will meet
below. It defines its struct pt_regs as
struct pt_regs {
struct uml_pt_regs regs;
};
and it's up to the architecture to define struct uml_pt_regs. This is
done this way because some UML userspace code needs to refer to
register values. So, uml_pt_regs is the structure that is safe for
userspace code to look at, which pt_regs is for kernel code. Access
macros for pt_regs therefore just call the equivalent access macro for
uml_pt_regs, like
#define PT_REGS_IP(r) UPT_IP(&(r)->regs)
and these should be defined in arch/um/include/sysdep/ptrace.h, which
should be able to be included in both user and kernel code. This
means it can't include either libc headers or kernel headers, but may
include safe UML headers.
Architecture headers
There are three headers, which go in arch/um/include/sysdep-$(SUBARCH),
which need to be written, and each needs to
contain a certain set of definitions.
frame.h
One of the first things that UML does when it boots is it creates and
saves a signal frame. This will be used when it delivers signals to
its processes. It will be copied onto the process stack and the data
in the original frame, like the signal number, sigcontext contents,
and restorer will be replaced. UML knows how to replace this stuff
because it figured out where in the stack frame it is. And it did
that by having the signal handler record the address of the signal,
the sigcontext structure, and a few other things.
This is done in a mostly architecture-independent way, but a little
help is needed from architecture-specific code. That code goes into
frame.h.
What's needed here are definitions of:
static inline unsigned long frame_restorer(void)
This returns the location on the stack of the signal state restorer.
On i386, this is the return address, which is next to the frame
pointer, so the i386 definition is
static inline unsigned long frame_restorer(void)
{
unsigned long *fp;
fp = __builtin_frame_address(0);
return((unsigned long) (fp + 1));
}
static inline unsigned long frame_sp(void)
This returns the value of the stack pointer when the signal handler is
first entered. Note that this is not necessarily the same value as
when the signal handler code is executing because it may have been
adjusted for local variables.
On i386, this is four bytes more than the frame pointer, so its
definition is
static inline unsigned long frame_sp(void)
{
unsigned long *fp;
fp = __builtin_frame_address(0);
return((unsigned long) (fp + 1));
}
In addition, there may be a need for the architecture to save more
information from the signal frame. There are two pairs of structures
and procedures which allow you to do this. The first pair are
expected to record raw addresses from the frame:
The arch_frame_data_raw may contain anything you want and
setup_arch_frame_raw is expected to fill it in. Both may be empty.
i386 needs to know the size of the floating point state that's on the
signal frame, so these save the address of the beginning of the sigcontext
structure, where the floating point state ends:
Then a similar structure and function pair is used to process the raw
addresses into something that's usable later. The i386 code assumes
that the floating point state runs from the top of the stack (which is
alone on its own page, so the top of the stack is the end of the page)
to the start of the sigcontext structure:
ptrace.h deals with the machine's register set. It defines the
following:
struct uml_pt_regs
which should contain a system call number, a set of system call
arguments, a flag saying whether the kernel was entered from userspace
or kernelspace, and a pointer to the sigcontext structure on the
current stack. The i386 definition is
struct uml_pt_regs {
unsigned long args[6];
long syscall;
int is_user;
void *sc;
};
which may in fact be architecture-independent. The only thing that
may need changing is the size of args[].
EMPTY_UML_PT_REGS
which is an initializer for uml_pt_regs. The i386 definition is
which returns the value of the appropriate register from the saved
sigcontext.
UPT_SET(regs, reg, val)
which sets the value of the appropriate register in the saved
sigcontext to whatever value is passed in. The i386 definitions of
these are big switch statements
#define UPT_REG(regs, reg) \
({ unsigned long val; \
switch(reg){ \
case EIP: val = UPT_IP(regs); break; \
case UESP: val = UPT_SP(regs); break; \
case EAX: val = UPT_EAX(regs); break; \
case EBX: val = UPT_EBX(regs); break; \
case ECX: val = UPT_ECX(regs); break; \
case EDX: val = UPT_EDX(regs); break; \
case ESI: val = UPT_ESI(regs); break; \
case EDI: val = UPT_EDI(regs); break; \
case EBP: val = UPT_EBP(regs); break; \
case ORIG_EAX: val = UPT_ORIG_EAX(regs); break; \
case CS: val = UPT_CS(regs); break; \
case SS: val = UPT_SS(regs); break; \
case DS: val = UPT_DS(regs); break; \
case ES: val = UPT_ES(regs); break; \
case FS: val = UPT_FS(regs); break; \
case GS: val = UPT_GS(regs); break; \
case EFL: val = UPT_EFLAGS(regs); break; \
default : \
panic("Bad register in UPT_REG : %d\n", reg); \
val = -1; \
} \
val; \
})
#define UPT_SET(regs, reg, val) \
do { \
switch(reg){ \
case EIP: UPT_IP(regs) = val; break; \
case UESP: UPT_SP(regs) = val; break; \
case EAX: UPT_EAX(regs) = val; break; \
case EBX: UPT_EBX(regs) = val; break; \
case ECX: UPT_ECX(regs) = val; break; \
case EDX: UPT_EDX(regs) = val; break; \
case ESI: UPT_ESI(regs) = val; break; \
case EDI: UPT_EDI(regs) = val; break; \
case EBP: UPT_EBP(regs) = val; break; \
case ORIG_EAX: UPT_ORIG_EAX(regs) = val; break; \
case CS: UPT_CS(regs) = val; break; \
case SS: UPT_SS(regs) = val; break; \
case DS: UPT_DS(regs) = val; break; \
case ES: UPT_ES(regs) = val; break; \
case FS: UPT_FS(regs) = val; break; \
case GS: UPT_GS(regs) = val; break; \
case EFL: UPT_EFLAGS(regs) = val; break; \
default : \
panic("Bad register in UPT_SET : %d\n", reg); \
break; \
} \
} while (0)
In addition to whatever macros you call from any additional PT_REGS_*
macros you define, there are a few that you definitely need
equivalents to. These will generally call into sigcontext macros
since they need to modify the current sigcontext.
UPT_SET_SYSCALL_RETURN
This sets the system call return value
UPT_RESTART_SYSCALL
This does whatever is necessary to make sure that the current system
call will restart when userspace is entered. Backing up the IP so
that it points at the system call instruction is probably enough.
UPT_ORIG_SYSCALL
This is the original location of the system call number. i386 moves
it from EAX to ORIG_EAX, so it refers to EAX. This is used when
restarting a system call to restore the registers to their original
values.
UPT_SYSCALL_NR
This pulls the system call number from the uml_pt_regs. On i386, it
comes directly from the uml_pt_regs structure. On other
architectures, it may make sense to get it from the sigcontext.
UPT_SYSCALL_RET
This returns the system call return value.
ptrace_user.h
This file defines a set of access macros into the hosts's pt_regs
structure. This is purely a userspace file which is used by parts of
UML which use ptrace to pull the process registers from the host
kernel and need to interpret them. These are the definitions that
should be here, and they should be implemented in terms of register
definitions in the host <asm/ptrace.h>.
PT_SYSCALL_NR
PT_SYSCALL_NR_OFFSET
The pt_regs index and ptrace offset of the system call number
PT_SYSCALL_ARG1_OFFSET
PT_SYSCALL_ARG2_OFFSET
PT_SYSCALL_ARG3_OFFSET
PT_SYSCALL_ARG4_OFFSET
PT_SYSCALL_ARG5_OFFSET
The offsets of the system call arguments
PT_SYSCALL_RET_OFFSET
The offset of the system call return value
PT_IP
PT_IP_OFFSET
The pt_regs index and offset of the IP
PT_SP
The index of the stack pointer
FRAME_SIZE
FRAME_SIZE_OFFSET
If the host pt_regs doesn't define FRAME_SIZE, set it to the number of
general purpose registers. Set FRAME_SIZE_OFFSET to the maximum
offset for PTRACE_GETREGS.
FP_FRAME_SIZE
FPX_FRAME_SIZE
These are the number of floating point registers and extended floating
point registers, respectively. The second is likely x86-specific. If
you don't define UM_HAVE_GETFPREGS/UM_HAVE_SETFPREGS or
UM_HAVE_GETFPXREGS/UM_HAVE_SETFPXREGS (see below), you can leave the
corresponding _FRAME_SIZE undefined.
UM_HAVE_GETREGS
UM_HAVE_SETREGS
UM_HAVE_GETFPREGS
UM_HAVE_SETFPREGS
UM_HAVE_GETFPXREGS
UM_HAVE_SETFPXREGS
These should be defined if the architecture defines PTRACE_GETREGS,
PTRACE_SETREGS, PTRACE_GETFPREGS, PTRACE_SETFPREGS, PTRACE_GETFPXREGS,
PTRACE_,SETFPXREGS respectively.
sigcontext.h
sigcontext.h defines a few sigcontext-related macros. Some of them
are accessed through the PT_REGS and UPT_REGS macros, so their
details are up to you. You will have to define similar things at the
very least, so here's what i386 defines
SC_RESTART_SYSCALL
Does whatever IP fiddling is needed to cause the current system call
to restart when userspace is re-entered. On i386, this just subtracts
2 from the IP since a system call instruction is 2 bytes long.
SC_SET_SYSCALL_RETURN
Sets the system call return value. On i386, this just sets %eax. On
ppc, it does a little more than that.
SC_FAULT_ADDR
SC_FAULT_WRITE
These two are called from generic UML code, so you have to implement
these as described. On a segfault, they pick out from a sigcontext
the fault address and whether the fault was on a write.
SEGV_IS_FIXABLE
This evaluates to non-zero if a segfault is one that can be fixed by
mapping in a page or changing page protections. If not, then it
returns zero, and the faulting process will simply be segfaulted.
This is called from generic UML code.
SC_START_SYSCALL
This is a general hook that's called at the start of a system call
before ptrace gets to see it. On x86, strace expects %eax to contain
-ENOSYS when it sees a system call entry. So, this macro sets that.
On other architectures, this may do nothing. This is called from
generic UML code.
void sc_to_regs(struct uml_pt_regs *regs, struct sigcontext *sc,
unsigned long syscall)
This is called at the beginning of a system call to fill in a pt_regs
structure from the sigcontext. It parses the system call from the
sigcontext and fills in the system call number and the arguments in
the pt_regs. This is called from generic UML code.
syscalls.h
syscalls.h defines any architecture-specific system calls. It does so
by defining
ARCH_SYSCALLS
which is a set of array element initializers which will be
included in the initialization of the system call table. The i386
ARCH_SYSCALLS looks in part like this
LAST_ARCH_SYSCALL
to be the last initialized element defined by ARCH_SYSCALLS.
This is used to fill the end of the system call table properly.
In addition, syscalls.h defines
EXECUTE_SYSCALL(syscall, regs)
which calls into a system call entry. The i386 pt_regs has been
arranged so that it can just be dumped on the stack and the right
thing will happen
The actual implementation of the port is contained in sys-$(SUBARCH).
You have complete freedom in this directory, except that when it is
built, it must produce an object file named sys.o which contains all
the code required by the generic kernel.
Here is a list of the files used by the existing ports, along with
what they define.
ptrace.c
int putreg(struct task_struct *child, unsigned long regno, unsigned long value)
This does any needed validity checking on the register and the value,
and assigns the value to the appropriate register in
child->thread.process_regs. If it fails, it returns -EIO.
This may be changed in the future so that it is provided with just the
register set rather than the whole task structure.
unsigned long getreg(struct task_struct *child, unsigned long regno)
getreg fetches the value of the requested register from
child->thread.process_regs, doing any required masking of registers
which don't use all their bits. This may also be changed to take the
register set rather than the task structure.
ptrace_user.c
Linux doesn't implement PTRACE_SETREGS and PTRACE_GETREGS on all
architectures. This file contains definitions of ptrace_getregs and
ptrace_setregs to hide this difference from the generic code.
Architectures which define PTRACE_SETREGS and PTRACE_GETREGS will
implement these functions as follows
Architectures which don't will implement them as loops which call
ptrace(PTRACE_GETREG, ...) or ptrace(PTRACE_SETREG, ...) for each register.
semaphore.c
This implements the architecture's semaphore primitives. It is highly
recommended to steal this from the underlying architecture by having
the Makefile make a link from arch/$(SUBARCH)/kernel/semaphore.c to
arch/um/sys-$(SUBARCH)/semaphore.c.
checksum.c or checksum.S
This implements the architecture's ip checksumming. This is
stolen from the underlying architecture in the same manner as
semaphore.c.
sigcontext.c
This defines a few sigcontext-related functions
int sc_size(void *data)
How big is a sigcontext? On x86, this takes the floating point state
into account as well as just the sigcontext structure itself.
int copy_sc_to_user(void *to_ptr, void *from_ptr, void *data)
Copies a sigcontext to a process stack. It must use copy_to_user, not
memcopy.
int copy_sc_from_user(void *to_ptr, void *from_ptr, void *data)
Copies a sigcontext from a process stack into kernel memory.
Similarly, this must use copy_from_user, not memcopy.
void sc_to_sc(void *to_ptr, void *from_ptr)
This copies a sigcontext from one kernel stack to another. It is used
during thread creation (kernel_thread(), fork(), or clone()) to
initialize a kernel stack with a signal frame that the new process can
return from.
sysrq.c
This needs to define
void show_regs(struct pt_regs *regs)
For the benefit of the SysRq driver.
Other files
If Linux on your architecture defines any private system calls, you
will need to implement them here. Normally, you can take the code
from the underlying architecture, and you might get away with linking
to the files in the other architecture that implement them.
Debugging the new port
There's no algorithm for doing this stage of the port, so I'll just
describe a number of useful tricks.
gdb is available. Use it. It's usable for any part of the kernel
after the beginning of start_kernel. If you need to debug anything
before that, the 'debugtrace' option is handle. It causes the tracing
thread to stop and wait to be attached with gdb. Then you can step
through the very early boot before start_kernel.
If you're post-mortem-ing a bug and you want to see what just happened
inside UML, there are some arrays which store some useful recent
history:
signal_record - stores the last 1024 signals seen by the tracing
thread, including the host pid of the process getting the signal, the
time, and the IP at which the signal happened. This is a circular
buffer and the latest entry is at index signal_index - 1.
syscall_record - the same, except it stores process system calls. It
stores the system call number, the return value (and 0xdeadbeef is
stored there if it hasn't returned), the UML pid of the process, and
the time. It is indexed by syscall_index, so the most recent entry is
at index syscall_index - 1.
These provide a decent picture of what UML has been doing lately.
Looking for unusual things here immediately before a bug happened is a
useful debugging technique. Correlating timestamps between the two
arrays is also sometimes useful.
If you have reproducable memory corruption, an extremely useful way to
track it down is to set the page that it happens on read-only and to
see what seg faults when it tries writing to that page. Obviously,
this only works if there aren't legitimate writes happening to that
page at the same time.
Hosted at
user-mode-linux-doc-20060501/case-studies.html 0000644 0000000 0000000 00000047375 12742461304 015741 0 ustar
UML in the real world
The purpose of this section is to let UML users describe what they're
doing with it and how they're doing it. This eventually (when there's
a wider variety of cases here) will let everyone see the breadth of
UML applications, and provide people who are interested in a specific
use with the information needed to implement it.
If you have a use for UML that you'd like to share, write it up,
including the following information
the problem that you're solving - and saying "I'm doing this wierd
thing because I can" is perfectly acceptable. That's more or less
how UML came into existence.
how you're using UML to solve it
how well it's working, including solutions or workarounds to any
problems you encountered
This is intended to be a collection of HOWTO material at least as much
as it's intended to be an advertisement for UML, so it would
definitely be a bonus to include HOWTO-like step-by-step instructions.
It could either be hosted on this site or linked from here back to
your site.
UML as an augmented firewall
From : Jon Wright (jon at gate dot sinica dot edu dot tw)
Date : 7 Sep 2001
I work for Prof Carmay Lim in the Institute of Biomedical Sciences,
Academia Sinica, Taipei, Taiwan
We're a structural and bioinfomatics group.
Basically on site we have about 10 Linux workstations, a 30 cpu beowulf,
and a couple of nfs fileservers all ip-masqed behind a single firewall.
We also have a group at National ChingHwa university at another city. We
need to provide access to our beowulf and other services to the students
but we don't want to allow direct connections to our firewall. Instead of
having direct connections to the firewall which if cracked gives access to
the internal network device and hence allows network sniffers, we boot a
UML kernel on the firewall itself and, using the slip networking (this was
set up 6 months ago - now we are looking at the tap interface), network the
UML kernel.
The host firewall accepts no syn packets at all on its external
IP address. All syn packets must be directed to the UML kernel IP address
to be accepted.
The UML system offers sshd, hhtpd, anon-ftp (oftpd) to everywhere, the
firewall only offers sshd and squid to the internal network and nothing to
the external network.
Students use ssh to logon on to the UML kernel using a generic name such
as user001, From there they can issue a second ssh command to the
internal network machines (ssh -l fred 192.168.0.140). We don't use the
same usernames or passwords on the UML system as the internal systems so
if someone does get UML account details, it won't help too much for
guessing usernames/passwords for the internal machines. We don't even list
the internal machines in the hosts file.
The firewall itself only allows connections from the UML machine to
internal machines on port 22 (set with ipchains) so if someone cracks the
UML machine they can not portscan the internal machines. The UML machine
does not contain a compiler, and many files/executables such as who, w,
ping, traceroute are read/write/execute for root only. We run tripwire
every night and email the report out, but we don't allow incoming email. If
fact, the host firewall only allows connections to the UML machine on
needed ports such as 22,80,21
The main purpose of the UML system is to provide a secure restricted
machine that offers limited external services and onward ssh connections
to our internal machines but does not allow any access to the network
devices themselves. (We don't allow loadable modules and we don't compile
hostfs)
Also being a 700mb file we can keep a compressed root file on cdrom and
use it to compare to the live one now and again and if need be restore
things using the host kernel.
All in all this is working very well for us at the moment. In fact, the
hardest part is organising all the ipchains rules on the host system so
that we limit what type of connections go where. For that we found
excellent help in the ipchains-howto, they used about 4 different
machines to provide firewall and external services while we saved on
hardware using one machine that worked as two. While other people can
probably pick some holes in this setup (I am not a pro ssyadmin) I haven't
seen any glaringly big holes - we have to allow the students access
somehow and having the external services on a system that does not have
direct access to hardware is a big bonus.
Many thanks for such a great tool
An implementation of a teaching network with UML
Virtual Network Laboratory - Christchurch Polytechnic Institute of
Technology (CPIT), New Zealand: A detailed Case Study HowTo.
William McEwan (Scottish exile), School of Computing, CPIT, New
Zealand.
mcewanw at hermes dot chchpoly dot ac dot nz
8 September 2001
"A problem with teaching data communications in an educational
institution is that there is always an inherent danger of data comms
experiments interfering with the normal operation of the campus
network. Many institutions have traditionally simply avoided much in
the way of practical data comms laboratory work. With the growing
importance of internetworking in general this is obviously an
unsatisfactory situation. With campus network infrastractures already
in place and centralised administration of IT established, it often
proves difficult (and expensive) to implement new network laboratories
that are sufficiently flexible and sufficiently isolated from the
normal campus".
[The above is extracted from a paper presented by this author: McEwan,
W. (2001) "Using Academic Research Methodologies to Improve the
Quality of Teaching: A Case Study". In Proc. Fourteenth Annual
Conference of the NACCQ, Napier, New Zealand: 83-93]
Introduction
With the above problems in mind, I am in the process of creating a
virtual network laboratory, using uml virtual machines. This work, at
the School of Computing, CPIT, NZ, is one of the key components in our
implementation of a "data comms and operating systems" teaching and
research laboratory. The uml configuration currently in active use in
our data comms teaching is illustrated below
(text version):
The virtual network laboratory is implemented on a 1 GHz Pentium III
system having 384 MBytes of RAM and a 20 GByte hard disk. The host
operating system is Redhat Linux 7.1 with a 512 MB swap partition. In
all, the virtual net consists of 20 virtual hosts sitting on 10
(sub)subnets connecting via one virtual router to our campus network
(and thence out to the Internet) as shown. The CPIT campus has been
assigned a class B address space which is subnetted into class C
address ranges (i.e. subnet mask=255.255.255.0). One of these /24
subnets has been allocated for this virtual lan server. On the
virtual network side of this lan the range is further /28 subnetted as
shown (i.e. netmask=255.255.255.240).
The current implementation uses the small debian uml root_fs. Using
debian package manager (dpkg) I have additionally installed telnetd so
that students can log in remotely. On the real host we have the
mindterm ssh client applet served by a running apache web server to
allow ssh login to that machine. We also have the free weirdx X
server applet served from the same machine. It is a great combination!
I shortly intend experimenting with X and ssh using the uml Linux
RH7.1 pristine root_fs in place of the small debian one.
One advantage of the above configuration, where one of the virtual
machines is used as a router, is that that machine can be configured
as a firewall effectively sandboxing the virtual network users into
the virtual lab (whilst allowing inward telnet or ssh traffic and
anything out as desired).
For quite a while, the FreeS/WAN project has been using virtual
networks of UMLs to test their code. Michael Richardson gave a talk
at OLS 2002 on what they're doing and how they're doing it. The
slides are available at
http://www.sandelman.ca/SSW/freeswan/fsumltesting/
.
Hosted at
user-mode-linux-doc-20060501/compile.html 0000644 0000000 0000000 00000053262 12742461304 014770 0 ustar
Building from source
Compiling the user mode kernel is just like compiling any other
kernel. Let's go through the steps, using 2.4.0-prerelease (current
as of this writing) as an example:
Download the latest UML patch from
the download page
In this example, the file is uml-patch-2.4.0-prerelease.bz2.
Run your favorite config; `make xconfig ARCH=um' is the most
convenient. `make config ARCH=um' and 'make menuconfig ARCH=um' will
work as well. The defaults will give you a useful kernel. If you
want to change something, go ahead, it probably won't hurt anything.
Note: If the host is configured with a 2G/2G address space split
rather than the usual 3G/1G split, then the packaged UML binaries will
not run. They will immediately segfault. See
this page for the scoop on
running UML on your system.
Finish with `make linux ARCH=um': the result is a file called `linux'
in the top directory of your source tree.
You may notice that the final binary is pretty large (many 10's of
megabytes for a debuggable UML). This is almost entirely symbol
information. The actual binary is comparable in size to a native
kernel. You can run that huge binary, and only the actual code and
data will be loaded into memory, so the symbols only consume disk
space unless you are running UML under gdb. You can strip UML:
host% strip linux
to see the true size of the UML kernel.
Make sure that you don't build this
kernel in /usr/src/linux. On some distributions, /usr/include/asm
is a link into this pool. The user-mode build changes the other end
of that link, and things that include <asm/anything.h> stop compiling.
The sources are also available from cvs. You can browse the CVS pool
or access it anonymously via
If you get the CVS sources, you will have to check them out into an
empty directory. You will then have to copy each file into the corresponding
directory in the appropriate kernel pool.
If you don't have the latest kernel pool, you can get the corresponding
user-mode sources with
host% cvs co -r v_2_3_x linux
where 'x' is the version in your pool. Note that you will not get the bug
fixes and enhancements that have gone into subsequent releases.
If you build your own kernel, and want to boot it from one of the
filesystems distributed from this site, then, in nearly all cases,
devfs must be compiled into the kernel and mounted at boot time. The
exception is the tomsrtbt filesystem. For this,
devfs must either not be in the kernel at all, or "devfs=nomount" must
be on the kernel command line. Any disagreement between the kernel
and the filesystem being booted about whether devfs is being used will
result in the boot getting no further than single-user mode.
If you don't want to use devfs, you can remove the need for it from a
filesystem by copying /dev from someplace, making a bunch of
/dev/ubd devices:
UML#
for i in 0 1 2 3 4 5 6 7; do mknod ubd$i b 98 $[ $i * 16 ]; done
and changing /etc/fstab and /etc/inittab to refer to the non-devfs devices.
Compiling and installing kernel modules
UML modules are built in the same way as the native kernel (with the
exception of the 'ARCH=um' that you always need for UML):
host% make modules ARCH=um
Any modules that you want to load into this kernel need to
be built in the user-mode pool. Modules from the native kernel won't
work. If you notice that the modules you get are much larger than
they are on the host, see the note above about the size of the final
UML binary.
You can install them by using ftp or something to copy them into the
virtual machine and dropping them into /lib/modules/`uname -r`.
You can also get the kernel build process to install them as
follows:
with the kernel not booted, mount the root filesystem in the top level
of the kernel pool:
host% mount root_fs mnt -o loop
run
host%
make modules_install INSTALL_MOD_PATH=`pwd`/mnt ARCH=um
unmount the filesystem
host% umount mnt
boot the kernel on it
If you can't mount the root filesystem on the host for some reason
(like it's a COW file), then an alternate approach is to mount the
UML kernel tree from the host into the UML with hostfs and run the modules_install inside
UML:
With UML booted, mount the host kernel tree inside UML at the same
location as on the host:
UML# mount none -t hostfs path to UML pool -o
path to UML pool
Run make modules_install:
UML# cd path to UML pool ; make modules_install
The depmod at the end may complain about unresolved symbols because
there is an incorrect or missing System.map installed in the UML
filesystem. This appears to be harmless. insmod or modprobe should
work fine at this point.
When the system is booted, you can use insmod as usual to get the modules
into the kernel. A number of things have been loaded into UML as
modules, especially filesystems and network protocols and filters, so
most symbols which need to be exported probably already are. However,
if you do find symbols that need exporting, let us know, and they'll be "taken care of".
If you try building an external module against a UML tree, you will
find that it doesn't compile because of missing includes. There are
less obvious problems with the CFLAGS that the module Makefile or
script provides which would make it not run even if it did build. To
get around this, you need to provide the same CFLAGS that the UML
kernel build uses.
A reasonably slick way of getting the UML CFLAGS is
cd uml-tree ; make script 'SCRIPT=@echo $(CFLAGS)' ARCH=um
If the module build process has something that looks like
$(CC) $(CFLAGS) file
then you can define CFLAGS in a script like this
CFLAGS=`cd uml-tree ; make script 'SCRIPT=@echo $(CFLAGS)' ARCH=um`
and like this in a Makefile
CFLAGS=$(shell cd uml-tree ; make script 'SCRIPT=@echo
$$(CFLAGS)' ARCH=um)
Compiling and installing uml_utilities
Many features of the UML kernel require a user-space helper program,
so a uml_utilities package is distributed separately from the kernel
patch which provides these helpers. Included within this is:
port-helper - Used by consoles which connect to xterms or ports
tunctl - Configuration tool to create and delete tap devices
uml_net - Setuid binary for automatic tap device configuration
uml_switch - User-space virtual switch required for daemon transport
The uml_utilities tree is compiled with:
host#
make && make install
Note that UML kernel patches may require a specific version of the
uml_utilities distribution. If you don't keep up with the mailing lists,
ensure that you have the latest release of uml_utilities if you are
experiencing problems with your UML kernel, particularly when dealing
with consoles or command-line switches to the helper programs
Changing the configuration of a virtual machine between boots can be
inconvenient because of the need to change config files in the root
filesystem that boots the machine. For example, if you want to change
the filesystems that are mounted, you need to change the kernel
command line to pass in the new filesystems, but you also need to
change /etc/fstab to mount them.
You can mount loopback-mount the root filesystem and go in and edit
the relevant files, but this is more manual than necessary, and it's
impossible without root privileges on the host.
The solution that I'm using is to:
Generate the config files and whatever else needs to be changed or
added to the filesystem that will boot the machine
Lay them out in a skeletal filesystem structure reflecting where
they will go in the virtual machine
Tar up that directory
Add that tar file as one of the block devices on the kernel
command line
Also add
CONFIG_DEV=device-name
to the
command line
The first thing the machine does after it mounts the root filesystem
read-write is check the CONFIG_DEV environment variable. If it has a
value, then it tars the new files from the device after saving the
originals. This is early enough to ensure that almost any new
configuration options will take effect.
To enable this, you need to put this script in
/etc/rc.d/rc.configfiles
#!/bin/sh
for ARG in $@
do
case $ARG in
start)
if [ "$CONFIG_DEV" != "" ]; then
restore=""
remove=""
for file in `tar tf $CONFIG_DEV`; do
if [ -e $file ]
then restore="$restore $file"
else remove="$remove $file"
fi
done
[ "$restore" != "" ] && tar cf /tmp/restore.tar $restore
echo $remove > /tmp/remove
tar xf $CONFIG_DEV
fi
;;
stop)
if [ "$CONFIG_DEV" != "" ]; then
[ -e /tmp/restore.tar ] && tar xf /tmp/restore.tar
[ -f /tmp/remove ] && rm -rf `cat /tmp/remove`
rm -rf /tmp/restore.tar /tmp/remove
fi
;;
*)
echo "usage: $0 (start|stop)"
exit 1
;;
esac
done
exit 0
and add a call to /etc/rc.S:
/etc/rc.d/rc.configfiles start
and a similar call to /etc/rc.0:
/etc/rc.d/rc.configfiles stop
The addition to /etc/rc.S needs to be immediately after the read-write
remount of /:
mount -o remount /
and the /etc/rc.0 needs to be just before the read-only remount of /:
mount -n -o remount,ro /
With this in place, it is possible to arbitrarily reconfigure a
virtual machine without booting it up or loopback mounting
its root filesystem beforehand.
Hosted at
user-mode-linux-doc-20060501/contacts.html 0000644 0000000 0000000 00000027670 12742461304 015162 0 ustar
Getting in touch with UML people
These are generally the preferred means to talk about or ask questions
about UML. There are two of them, uml-user
(user-mode-linux-user (at) lists.sourceforge.net -
subscription info) and uml-devel
(user-mode-linux-devel (at) lists.sourceforge.net -
subscription info). The difference
between them is what you'd expect. uml-user is for people who are
trying to make this darned thing do what you want and for those who
are trying to help. uml-devel is more for discussion of the code. I post
announcements of new patches and CVS updates to uml-devel, and
announcements of full releases to uml-user.
Send bug reports to either list, whichever is more convenient.
The guy in charge
That would be me, Jeff Dike, jdike (at) karaya (dot) com. If
something's not suitable for one of the lists, or you're just shy,
then send it to me personally.
UML users have started to build their own tools and infrastructure
around UML in order to make it better suited for their own
applications. This page lists the ones that I know of so that people
who need something similar don't necessarily need to go off and
reinvent the wheel. If you have something (or know of something) that
should be listed, let us
know.
DNS root filesystem from Chris Reahard
This filesystem is a small filesystem intended to run a jailed named
and nothing else. This general area, of using a UML virtual machine
to jail a single service, is a ripe one, and I welcome contributions
of other filesystems that are specialized for other services.
It's available from the jail
filesystems section of the download
page. Chris' description is available
here .
BusyBox root filesystem builder from Julien Gaulmin
This is a little package which automates the building of a
BusyBox/uClibc root filesystem for UML. Download
this tarball, uncompress it, untar it, and read the README and
Makefile.
UMLd - A UML management daemon from David Coulson
A system with lots of UMLs can be a management nightmare, so David
Coulson has written a daemon to manage all of his UMLs, allowing UML
users to perform functions on their UML via a web interface. For a
network running lots of UML instances on machines, UMLd makes
housekeeping much easier. More information on UMLd, along with
downloads, can be found
here.
Hosted at
user-mode-linux-doc-20060501/cpit.html 0000644 0000000 0000000 00000115464 12742461304 014302 0 ustar
CPIT Case Study Cont'd
Building such a network, involving such subnetting is quite complex.
I have therefore written some shell scripts to build the network
almost automatically. The complexity of the scripts is such that it
would have been easier to implement them using awk or perl or such
like but currently they are in plain bash syntax. They have been
rapidly prepared so are far from elegant, but being in plain bash
syntax has the advantage that more people are likely to understand
them. The scripts are quite flexible - any values for the subnet
x.y.z can be given as a parameter (though the current script is fixed
for a sub(sub)net mask of 225.255.255.240).
[ The scripts below are available in a tarball
here - jdike]
Rather than going into long discussions of how the setup is
implemented I have instead carefully commented the scripts themselves
in explanation.
One of the most important things in the design is to decide on a
suitable directory structure including the permissions used in that
structure.
1. I have therefore created a user "uml" with a home directory /home/uml as the principal operator of the build scripts.
The arrangement chosen requires that ALL scripts and the root_fs MUST
be stored in this /home/uml directory. The cow files however are kept
in separate directories (one per virtual machine). This helps protect
them from other users.
2. Each virtual machine is allocated a user name and home directory
for its cow file. The home directory for the virtual machines is a
sub-directory of /home/uml. The names of the "virtual-machine-users"
(and thus their home directories) are of the form: umlnn_hh where nn
is the subnet they lie on and hh the host number on that subnet. For
example, the virtual host with IP=x.y.z.17 would have the directory
and user name of uml16_17 (the value 16 for the subnet comes from the
relevant address table for a /28 subnetted network like this).
The uml virtual firewall/router has the name/directory of umlfw224_225
(i.e. its IP=x.y.z.225).
3i. Three main scripts are run, one after the other in order to build
the network. It is important that all scripts are run from the
directory /home/uml (i.e. ~uml).
The scripts are called: umlcreatenet1, umlcreatenet2, and umlcreatenet3.
The way it is done presently, umlcreatenet1 must be run by user root
(it insmods ethertap and runs the many uml_router occurrences etc).
3ii. For security reasons user root does not run the second script
umlcreatenet2. It is run instead by user "uml". This second script
boots up the virtual machines themselves. After running this second
script all machines should be running in their own xterm ready for
final network configuration.
3iii. The third script "umlcreatenet3" (which must be run by user
root) sets up the route on the real host towards the virtual network.
3iv. At this stage the main components of the network are already in
place (ethertap and the uml_router daemons etc). All that remains is
to arrange the ifconfig up and route commands on the virtual hosts
themselves. That is not yet fully automatic. Instead I have created
two special scripts (one for the virtual router, and one for the other
virtual hosts). These scripts have been copied into the /root
directory of the small debian root_fs (by entering:
mount root_fs [host_mount_point] -o loop (followed by cp'ing the
scripts into /root).
The two virtual host network configuration scripts are called
umlvfw.conf and umlvhost.conf.
Running any one of these scripts, without supplying any parameters,
results in a brief, but hopefully helpful, usage message.
In practice their are several more scripts than those outlined in the
simple startup steps above. The umlcreatenet(1,2,3) scripts are
actually only "call-up" scripts - they call up the underlying scripts
that do the real work of configuration. The underlying scripts are
called (and they are run in the following order by the umlcreatenet
scripts): umlkillall, umlkillnet (these first two are simply there to
brutally "clean up" any previous uml occurrencies), umlhostsetup1,
umlvfs, umlvhost, umlhostsetup2.
The best way of understanding the whole thing is simply to read the scripts through in the correct order. They are individually quite simple really!!
Summary:
a. Create the user uml (home=/home/uml), and (with home subdirectories
in /home/uml) users: umlfw224_225, uml16_17, uml16,18, uml32_33,
uml32_34 etc...
b. Put every script and the root_fs in /home/uml (and check all the
permissions are okay).
c. Then run the following scripts (and always from pwd of /home/uml) in this order:
umlcreatenet1 (with the parameters required indicated as usage when run)
umlcreatenet2
umlcreatenet3
d. Login to the uml virtual machines (as root) and immediately run the network script:
./umlfw.conf (for the virtual router) or ./umlvhost.conf (for any of the other virtual machines).
Here then are the scripts themselves, and an appendix showing the
permissions that were assigned to each of them and their relevant
directories. Hopefully this stuff will save somebody the many many
tens of hours it took me to put it all together :-).
I. The umlcreatenet1 script:
#!/bin/sh
#Program: umlcreatenet1
#Licence: GPL
#Author: William McEwan
#Date: 6 Sep 2001
# 1. This is the main script (part 1 of 3) for building the Virtual
#Network Laboratory semi-automatically.
# "umlcreatenet1 (as user root) creates a clean slate and sets up the
#host ethertap and uml_router daemons.
# 2. This script should be followed by running script
# ~uml/umlcreatenet2 (as user uml).
# 3. Finally the script ~uml/umlcreatenet3 should be run (as user root).
# 4. The only thing left to do after that it to run the conf scripts
# on the uml machines themselves to complete networking. Assuming a
# telnetd or sshd is present on the virtual machines it should then be
# possible to remotely log in to them.
# READ this script first to help you understand how the whole scheme works.
# It runs all the other scripts for you except for umlvfw.conf and for
# umlvhost.conf (which you run once the virtual machines have booted -
# refer to the related umlvfw and the umlvhost script for usage comments).
# I intend automating that last part later.
# If you have difficulty getting it to work smoothly, start by giving
# more access permissions to all relevant files and directories (and
# re-organise that later).
# IMPORTANT NOTES:
# NOTE 1: Run "./umlcreatenet1 subnet (e.g. 192.168.5)" as user root.
# All these scripts assume you want the (sub)subnet mask
# 255.255.255.240
# NOTE 2: root_fs MUST be placed in ~uml (i.e. uml home directory) for the
# scripts to find it and its modes must be -rw-rw---- root uml
# All the scripts (e.g. umlkillall, umlkillnet, umlhostsetup1,
# umlvfw, umlvhost, umlhostsetup2 MUST also be kept in ~uml
# for this script to find them etc) and all permissions must
# be carefully set up to allow user uml to access them.
# NOTE 3: Make sure you have previously created the uml user(s) along
# with their associated home directories.
# And that you have setup the the main text described permissions and
# groups etc.
# Each virtual machine must have its own directory created in
# ~uml. For example, vhost 192.165.5.17 in my set up has home
# directory /home/uml/uml16_17
# (You can use the -G and -b options of useradd for doing this kind of
# stuff) - in practice it is best to write an automating script for
# adding the uml users.
# The 16_17 stands for sub(subnet) 16 and host id 17. i.e. the number
# 16 usefully tells you which sub(subnet) this particular host is on
# (i.e. subnet 192.168.5.16/28).
cpitsubnet="$1"
case "$cpitsubnet" in
"") echo
echo 'usage: ./umlcreatenet1 subnet(e.g. 192.168.5)'
exit 1
;;
*) ;;
esac
#MAKE SURE you are root user before running this script for the following to work...
# Have a look at the internals of all the following scripts to see
# what they actually do...
cd ~uml
# Get rid off any old uml linux processes and network daemons.
# i.e. clean start:
./umlkillall
./umlkillnet
# Set up ethertap and the uml_router daemons on the host computer:
./umlhostsetup1 238 "$cpitsubnet"
II. The umlcreatenet2 script:
# Program: umlcreatenet2
#This is the second part of the startup script.
#NOTE WELL: It should be run as user uml.
#Boots up all the virtual machines starting with the firewall router
cd ~uml
cpitsubnet="$1"
case "$cpitsubnet" in
"") echo
echo 'usage: ./umlcreatenet2 subnet(e.g. 192.168.5)'
exit 1
;;
*) ;;
esac
./umlvfw 225 238 0 "$cpitsubnet"
./umlvhost 17
./umlvhost 18
./umlvhost 33
./umlvhost 34
./umlvhost 49
./umlvhost 50
./umlvhost 65
./umlvhost 66
./umlvhost 81
./umlvhost 82
./umlvhost 97
./umlvhost 98
./umlvhost 113
./umlvhost 114
./umlvhost 129
./umlvhost 130
./umlvhost 145
./umlvhost 146
./umlvhost 161
./umlvhost 162
III. The umlcreatenet3 script:
# Program: umlcreatenet3
#This is the third and final part of the startup script.
#NOTE WELL: It MUST be run as user root.
# Set up the final route from the host computer to the virtual network.
# Make sure you are in directory ~uml (where all these uml scripts are
# located):
cd ~uml
cpitsubnet="$1"
case "$cpitsubnet" in
"") echo
echo 'usage: ./umlcreatenet3 subnet(e.g. 192.168.5)'
exit 1
;;
*) ;;
esac
./umlhostsetup2 "$cpitsubnet"
# Should all be up now. Now you just need to run "umlvhost.conf on each
# booted uml host (and umlvfw.conf on the virtual firewall router).
IV. The rough and brutal umlkillall script:
# This is just an inelegant quick fix for killing off all the virtual machines
# It assumes "linux" is name of the uml process.
# I generally run this script as the first thing before building the
# virtual net (as user root).
# It, followed by running "umlkillnet", (as user root) cleans
# everything up before beginning the main virtual hosts build.
# I also tend to rm all the cow files before rebuilding the vnet.
# usage: ~uml/umlkillall
# NOTE: Instead of running this script individually it can be called
# by the script "~uml/umlcreatenet1"
# whose purpose is to call up all the other scripts as well. View
# umlcreatenet1 (2 and 3) for details.
kill -9 `ps aux | grep linux | cut -c10-14`
rm -rf /tmp/uml/*
V. The equally rough umlkillnet script:
# Just an inelegant quick fix for killing off all uml_router daemons
# and ethertaps ready for a clean start
# usage: ~uml/umlkillnet
# NOTE: Instead of running this script individually it can be called
# by the script "~uml/umlcreatenet1"
# whose purpose is to call up all the other scripts as well. View
# umlcreatenet1 (2 and 3) for details.
kill -9 `ps aux | grep uml_router | cut -c10-14`
rm -f 14*
ifconfig tap0 down
rmmod ethertap
rmmod ethertap0
VI. The similarly awful umlkillxterm script:
#Just an inelegant quick way to kill all xterms/vconsoles if you ever
#want to...
#usage: ~uml/umlkillxterm
kill -9 `ps aux | grep xterm | cut -c10-14`
VII. The umlhostsetup1 script:
#!/bin/sh
#Program: umlhostsetup1
#Licence: GPL
#Author: William McEwan
#Date: 6 Sep 2001
# This script sets up the host computer.
# Run it (as user root) just after the "clean up scripts" (i.e
# umlkillall followed by umlkillnet).
# NOTE: Instead of running this script individually it can be called
# by the script ~uml/umlcreatenet1
# whose purpose is to call up all the other scripts as well. View
# umlcreatenet1 (2 and 3) for details.
tap0ip="$1" # or in this example case could simply use: tap0ip="238"
# (since that is what it is here!)
case "$1" in
238) ;; # Could modify script to increase acceptable tap
# values: e.g. 23[1-8])
*) echo
echo 'usage: ./umlhostsetup1 tapip(e.g. 238) \
subnet(e.g. 192.168.5)'
exit 1
;;
esac
cpitsubnet="$2" # or could use: cpitsubnet="x.y.z"
tap0mac="10:0:0:0:0:2" # or could use: tap0mac=$3
# (or whatever MAC address you want for tap0)
#Just in case they need done:
mknod /dev/tap0 c 36 16
insmod ethertap #making tap0
#Bring tap0 up on host:
ifconfig tap0 hw ether "$tap0mac" arp mtu 1484 ${cpitsubnet}.${tap0ip} \
netmask 255.255.255.240 broadcast ${cpitsubnet}.239
#Create all the subnet "switches" (daemons):
uml_router -unix 14003 14004 &
uml_router -unix 14005 14006 &
uml_router -unix 14007 14008 &
uml_router -unix 14009 14010 &
uml_router -unix 14011 14012 &
uml_router -unix 14013 14014 &
uml_router -unix 14015 14016 &
uml_router -unix 14017 14018 &
uml_router -unix 14019 14020 &
uml_router -unix 14021 14022 &
uml_router -unix 14023 14024 &
uml_router -unix 14025 14026 &
uml_router -unix 14027 14028 &
uml_router -unix 14029 14030 &
#Give a wee bit time to assure uml_router daemon sockets are ready:
sleep 1
#Make sure the uml virtual hosts can access the uml_router daemon(s) sockets
chgrp uml 14*
chmod 770 14*
VIII. The umlvfw (virtual router/firewall machine) script:
#!/bin/sh
#Program: umlvfw
#Licence: GPL
#Author: William McEwan
#Date: 6 Sep 2001
# This script sets up the virtual machine being used as a router (firewall)
# between the host computer and the rest of the virtual machines.
# In this Case Study this one virtual router connects via ethertap
# (its eth0 IP=225) to the host computer (IP=238).
# The other nine interfaces on this virtual router connect to the other
# virtual machines - two such on each network segment.
# It is planned to set up a filtering firewall on this virtual router in
# order to help sandbox remote users inside the virtual network laboratory.
# "umlvfw" should be run as user uml (not as user root).
# Run it after running umlkillall, umlkillnet and umlhostsetup1 or as part
# of the general startup scripts (~uml/umlcreatenet1,2,3)
# that comes after these.
# NOTE: Instead of running this script individually it can be called
# by the script "~uml/umlcreatenet2" whose purpose is to call up all
# the other scripts as well. View umlcreatenet2 for details.
# (This virtual router is later connected to the uml_router daemon segments
# using the associated script: "umlvfw.conf".
# Note: I previously copy umlvhost.conf onto the root_fs (debian small),
# into the directory /root using:
# mount root_fs [host_dir_mountpoint] -o loop and then using cp.
# "umlvfw.conf is set up to connect to the uml_router daemons using:
# IP addresses 30,46,62,78,94,110,126,142,158,174.
# These are the highest host numbers for each of the segments on
# a 255.255.255.240 subnetted IP address).
# Also, once (and only once) the virtual machine has booted up,
# login as root and run the associated virtual machine configuration
# script "umlvfw.conf" by
# entering: ./umlvfw.conf 225 238 0
umask 006 #So that the cow files are created with the correct permissions
case "$1" in
22[5-9]|23[0-7]) subsubnet=224;;
*) echo
echo 'usage: ./umlvfw ip_address_last_octet(e.g. 225)
\
tapip(e.g. 238)'
echo ' tapnum(e.g. 0) subnet(e.g. 192.168.5)'
exit 1
;;
esac
mainip="$1" # or could use: mainip=225 (= other end of tap)
tapip="$2" # or could use: tapip=238
tapnum="$3" # e.g. tapnum=0 (makes it tap0 below)
cpitsubnet="$4" # or could use: cpitsubnet="x.y.z"
umlswitch1="14003,14004"
umlswitch2="14005,14006"
umlswitch3="14007,14008"
umlswitch4="14009,14010"
umlswitch5="14011,14012"
umlswitch6="14013,14014"
umlswitch7="14015,14016"
umlswitch8="14017,14018"
umlswitch9="14019,14020"
umlswitch10="14021,14022"
umlswitch11="14023,14024"
umlswitch12="14025,14026"
umlswitch13="14027,14028"
umlswitch14="14029,14030"
# Start up uml linux as a daemon (nohup) with one xterm to do the final
# network configuration; using umlvfw.conf (as described in that script):
cow_locat=~uml/umlfw${subsubnet}_${mainip}/cowfw${subsubnet}_${mainip}
nohup linux umid=${mainip} ubd0=${cow_locat},root_fs \
eth0=ethertap,"tap${tapnum}",,${cpitsubnet}.${tapip} \
eth1=daemon,,unix,${umlswitch1} eth2=daemon,,unix,${umlswitch2} \
eth3=daemon,,unix,${umlswitch3} eth4=daemon,,unix,${umlswitch4} \
eth5=daemon,,unix,${umlswitch5} eth6=daemon,,unix,${umlswitch6} \
eth7=daemon,,unix,${umlswitch7} eth8=daemon,,unix,${umlswitch8} \
eth9=daemon,,unix,${umlswitch9} eth10=daemon,,unix,${umlswitch10} \
eth11=daemon,,unix,${umlswitch11} eth12=daemon,,unix,${umlswitch12} \
eth13=daemon,,unix,${umlswitch13} eth14=daemon,,unix,${umlswitch14} \
ssl=pty con=pty con0=xterm &
umask 022 #returning umask to original values on my host
IX. The umlvhost (every other virtual machine) script:
#!/bin/sh
#Program: umlvhost
#Licence: GPL
#Author: William McEwan
#Date: 6 Sep 2001
# This script sets up a single virtual machine on
# the network segment relevant to its ip address (e.g ip 17,18...33,34...etc).
# It should be run as user uml (not as user root).
# Run it after running umlkillall, umlkillnet and umlhostsetup1 or
# as part of a general script (umlcreatenet1,2,3)that comes after these.
# NOTE: Instead of running this script individually it can be called by
# the script "~uml/umlcreatenet2" whose purpose is to call up all
# the other scripts as well. View umlcreatenet2 for details.
# (It is later connected to the uml_router daemon segments using
# the associated script: "umlvhost.conf".
# Note: I previously copy umlvhost.conf onto the root_fs (debian small),
# into the directory /root using:
# mount root_fs [host_dir_mountpoint] -o loop and then using cp).
# Also, once (and only once) the virtual machine has booted up,
# login as root and run the script "umlhost.conf" by entering,
# for example: ./umlvhost.conf 17
# Repeat the above for the other (non-firewall) virtual machines
# (using different IPs!).
# (Later I hope to make this networking more automatic..:-).
umask 006 #So that the cow files are created with the correct permissions
mainip="$1"
#Arrange for the host IP to end up on the correct uml_router daemon subnet:
case "$mainip" in
[1-9]|1[0-4]) subsubnet=0 ; umlswitch="14001,14002" ;;
1[7-9]|2[0-9]|30) subsubnet=16 ; umlswitch="14003,14004" ;;
3[3-9]|4[0-6]) subsubnet=32 ; umlswitch="14005,14006" ;;
49|5[0-9]|6[0-2]) subsubnet=48 ; umlswitch="14007,14008" ;;
6[5-9]|7[0-8]) subsubnet=64 ; umlswitch="14009,14010" ;;
8[1-9]|9[1-4]) subsubnet=80 ; umlswitch="14011,14012" ;;
9[7-9]|10[0-9]|110) subsubnet=96 ; umlswitch="14013,14014" ;;
11[3-9]|12[0-6]) subsubnet=112 ; umlswitch="14015,14016" ;;
129|13[0-9]|14[1-2]) subsubnet=128 ; umlswitch="14017,14018" ;;
14[5-9]|15[0-8]) subsubnet=144 ; umlswitch="14019,14020" ;;
16[1-9]|17[0-4]) subsubnet=160 ; umlswitch="14021,14022" ;;
17[7-9]|18[0-9]|190) subsubnet=176 ; umlswitch="14023,14024" ;;
19[3-9]|20[0-6]) subsubnet=192 ; umlswitch="14025,14026" ;;
209|21[0-9]|22[0-2]) subsubnet=208 ; umlswitch="14027,14028" ;;
# 22[5-9]|23[0-8]) subsubnet=224 ; umlswitch="14029,14030" ;;
24[1-9]|25[0-4]) subsubnet=240 ; umlswitch="14031,14032" ;;
*) echo
echo 'invalid ip_octet'
echo 'usage: ./umlvhost dotted_dec_ip_address_last_octet(e.g. 17)'
exit 1
;;
esac
# Start up uml linux as a daemon (nohup) with one xterm to do the
# final network configuration; using umlvhost.conf (as described in
# that script):
cow_locat=~uml/uml${subsubnet}_${mainip}/cow${subsubnet}_${mainip}
nohup linux umid=uml${mainip} ubd0=${cow_locat},root_fs \
eth0=daemon,,unix,${umlswitch} ssl=pty con=pty con0=xterm &
umask 022 #returning umask to original values on my host
X. The virtual router/firewall network configuration script (umlvfw.conf):
#!/bin/sh
#Program: umlvfw.conf
#Licence: GPL
#Author: William McEwan
#Date: 6 Sep 2001
# This script is used to finally connect the virtual router (firewall)
# to the virtual network.
# It is run from the virtual machine itself after it is first booted
# as root user.
# I plan to automate this better later.
# I store this script in the /root directory of the debian small
# root_fs used in the experiment.
# I similarly store one copy of the script umlvhost.conf in the same place.
# I get them there using the command:
# mount root_fs /some_host_mount_point -o loop
# followed by cp them from the host computer.
# In the current setup $1 should be entered on the
# commandline as 225 (= the virtual machine end of tap0):
case "$1" in
22[5-9]|23[0-7]) ;;
*) echo
echo 'usage: umlhost ip_address_last_octet(e.g. 225) \
tapip(e.g. 238) subnet(e.g. 192.168.5'
exit 1
;;
esac
mainip="$1" # e.g. mainip=225 (= other end of tap).
tapip="$2" # or could use: tapip=238 (= host computer tap0 IP address).
cpitsubnet="$3" # or could make: cpitsubnet="x.y.z"
# Configure the ethertap connection from eth0 to tap:
ifconfig eth0 ${cpitsubnet}.${mainip} netmask 255.255.255.240
broadcast ${cpitsubnet}.239 up
# Configure the virtual router connections to the uml_router daemon sockets:
ifconfig eth1 ${cpitsubnet}.30 netmask 255.255.255.240 broadcast \
${cpitsubnet}.31 up
ifconfig eth2 ${cpitsubnet}.46 netmask 255.255.255.240 broadcast \
${cpitsubnet}.47 up
ifconfig eth3 ${cpitsubnet}.62 netmask 255.255.255.240 broadcast \
${cpitsubnet}.63 up
ifconfig eth4 ${cpitsubnet}.78 netmask 255.255.255.240 broadcast \
${cpitsubnet}.79 up
ifconfig eth5 ${cpitsubnet}.94 netmask 255.255.255.240 broadcast \
${cpitsubnet}.95 up
ifconfig eth6 ${cpitsubnet}.110 netmask 255.255.255.240 broadcast \
${cpitsubnet}.111 up
ifconfig eth7 ${cpitsubnet}.126 netmask 255.255.255.240 broadcast \
${cpitsubnet}.127 up
ifconfig eth8 ${cpitsubnet}.142 netmask 255.255.255.240 broadcast \
${cpitsubnet}.143 up
ifconfig eth9 ${cpitsubnet}.158 netmask 255.255.255.240 broadcast \
${cpitsubnet}.159 up
ifconfig eth10 ${cpitsubnet}.174 netmask 255.255.255.240 broadcast \
${cpitsubnet}.175 up
# On the virtual router machine: add default route (gw = host tap0)
and turn on forwarding:
route add default gw ${cpitsubnet}.${tapip}
echo 1 > /proc/sys/net/ipv4/ip_forward
XI. The network configuration script for all the other virtual hosts
(umlvhost.conf):
#!/bin/sh
#Program: umlvhost.conf
#Licence: GPL
#Author: William McEwan
#Date: 6 Sep 2001
# This script is used to finally connect the virtual machines to the
# virtual network.
# It is run from the virtual machine itself after it is first booted
# as root user.
# I plan to automate this better later.
# I store this script in the /root directory of the debian small
# root_fs used in the experiment.
# I similarly store one copy of the script umlvfw.conf in the same place.
# I get them there using the command:
# mount root_fs /some_host_mount_point -o loop
# followed by cp them from the host computer.
cpitsubnet="$2" # or could use: cpitsubnet="x.y.z"
mainip="$1"
#Arrange for the host IP to end up on the correct uml_router daemon subnet:
case "$mainip" in
[1-9]|1[0-4]) gwip=14 ; bcastip=15 ;;
1[7-9]|2[0-9]|30) gwip=30 ; bcastip=31 ;;
3[3-9]|4[0-6]) gwip=46 ; bcastip=47 ;;
49|5[0-9]|6[0-2]) gwip=62 ; bcastip=63 ;;
6[5-9]|7[0-8]) gwip=78 ; bcastip=79 ;;
8[1-9]|9[1-4]) gwip=94 ; bcastip=95 ;;
9[7-9]|10[0-9]|110) gwip=110 ; bcastip=111 ;;
11[3-9]|12[0-6]) gwip=126 ; bcastip=127 ;;
129|13[0-9]|14[1-2]) gwip=142 ; bcastip=143 ;;
14[5-9]|15[0-8]) gwip=158 ; bcastip=159 ;;
16[1-9]|17[0-4]) gwip=174 ; bcastip=175 ;;
17[7-9]|18[0-9]|190) gwip=190 ; bcastip=191 ;;
19[3-9]|20[0-6]) gwip=206 ; bcastip=207 ;;
209|21[0-9]|22[0-2]) gwip=222 ; bcastip=223 ;;
22[5-9]|23[0-8]) gwip=238 ; bcastip=239 ;;
24[1-9]|25[0-4]) gwip=254 ; bcastip=255 ;;
*) echo
echo 'invalid ip_octet'
echo 'usage: ./umlvhost.conf ip_octet(e.g. 17) \
subnet(e.g.192.168.5)'
exit 1
;;
esac
# Configure the virtual machine eth0:
ifconfig eth0 ${cpitsubnet}.${mainip} netmask 255.255.255.240
broadcast \
${cpitsubnet}.${bcastip} up
# Add a route on the virtual machine back towards the host computer
# via the uml virtual firewall:
route add default gw ${cpitsubnet}.${gwip}
XII. Appendix A: The permissions on the files and directories etc.:
The following is the beginning of a gdb session with the kernel under
gdb from the beginning. It starts at the top of start_kernel() and
goes one line at a time through the initial kernel startup.
GNU gdb 4.17.0.11 with Linux support
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) att 1
Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1
0x1009f791 in __kill ()
(gdb) b start_kernel
Breakpoint 1 at 0x100ddf83: file init/main.c, line 515.
(gdb) c
Continuing.
Breakpoint 1, start_kernel () at init/main.c:515
515 printk(linux_banner);
(gdb) n
516 setup_arch(&command_line);
(gdb)
517 printk("Kernel command line: %s\n", saved_command_line);
(gdb)
518 parse_options(command_line);
(gdb)
519 trap_init();
(gdb)
520 init_IRQ();
(gdb)
521 sched_init();
(gdb)
522 time_init();
(gdb)
523 softirq_init();
(gdb)
530 console_init();
This is tiring, so I just let it continue booting.
(gdb) c
Continuing.
It's booted, so I ^C it to see what it thinks is up.
Program received signal SIGINT, Interrupt.
0x100a4bc1 in __libc_nanosleep ()
(gdb) bt
#0 0x100a4bc1 in __libc_nanosleep ()
#1 0x100a4b7d in __sleep (seconds=10) at ../sysdeps/unix/sysv/linux/sleep.c:78
#2 0x10095fbf in do_idle () at process_kern.c:424
#3 0x10096052 in cpu_idle () at process_kern.c:450
#4 0x100de0a4 in start_kernel () at init/main.c:593
#5 0x10098df2 in start_kernel_proc (unused=0x0) at um_arch.c:72
#6 0x1009858f in signal_tramp (arg=0x10098db8) at trap_user.c:50
(gdb)
It's busy sleeping in the idle loop. I'll set a breakpoint in the
scheduler and pick it up on the next context switch.
(gdb) b schedule
Breakpoint 2 at 0x10004acd: file sched.c, line 496.
(gdb) c
Continuing.
Breakpoint 2, schedule () at sched.c:496
496 if (!current->active_mm) BUG();
(gdb) bt
#0 schedule () at sched.c:496
#1 0x10095fb3 in do_idle () at process_kern.c:421
#2 0x10096052 in cpu_idle () at process_kern.c:450
#3 0x100de0a4 in start_kernel () at init/main.c:593
#4 0x10098df2 in start_kernel_proc (unused=0x0) at um_arch.c:72
#5 0x1009858f in signal_tramp (arg=0x10098db8) at trap_user.c:50
Here we are in the scheduler. I'll 'next' through the first few lines
of the scheduler, get bored, and set a breakpoint in the SIGIO
interrupt handler.
(gdb) n
497 if (tq_scheduler)
(gdb)
501 prev = current;
(gdb)
502 this_cpu = prev->processor;
(gdb)
504 if (in_interrupt())
(gdb)
510 if (softirq_state[this_cpu].active & softirq_state[this_cpu].mask)
(gdb)
518 sched_data = & aligned_data[this_cpu].schedule_data;
(gdb)
520 spin_lock_irq(&runqueue_lock);
(gdb) b sigio_handler
Breakpoint 3 at 0x10094fdc: file irq_user.c, line 36.
(gdb) c
Continuing.
Breakpoint 2, schedule () at sched.c:496
496 if (!current->active_mm) BUG();
Oops, that process scheduled back to the idle thread. Get rid of that
breakpoint and continue again.
(gdb) d 2
(gdb) c
Well, the SIGIO handler is waiting for something and nothing is
happening by itself. So, I'll type something at one of the login
prompts to wake it up.
Breakpoint 4, do_IRQ (irq=2, user_mode=0) at irq.c:266
266 irq_desc_t *desc = irq_desc + irq;
(gdb) n
271 regs.user_mode = user_mode;
(gdb)
272 kstat.irqs[cpu][irq]++;
(gdb)
274 desc->handler->ack(irq);
(gdb)
279 status = desc->status & ~(IRQ_REPLAY | IRQ_WAITING);
(gdb)
280 status |= IRQ_PENDING; /* we _want_ to handle it */
(gdb)
286 action = NULL;
(gdb)
287 if (!(status & (IRQ_DISABLED | IRQ_INPROGRESS))) {
(gdb) l
282 /*
283 * If the IRQ is disabled for whatever reason, we cannot
284 * use the action we have.
285 */
286 action = NULL;
287 if (!(status & (IRQ_DISABLED | IRQ_INPROGRESS))) {
288 action = desc->action;
289 status &= ~IRQ_PENDING; /* we commit to handling */
290 status |= IRQ_INPROGRESS; /* we are handling it */
291 }
(gdb) l
292 desc->status = status;
293
294 /*
295 * If there is no IRQ handler or it was disabled, exit early.
296 Since we set PENDING, if another processor is handling
297 a different instance of this same irq, the other processor
298 will take care of it.
299 */
300 if (!action)
301 goto out;
(gdb) l
302
303 /*
304 * Edge triggered interrupts need to remember
305 * pending events.
306 * This applies to any hw interrupts that allow a second
307 * instance of the same irq to arrive while we are in do_IRQ
308 * or in the handler. But the code here only handles the _second_
309 * instance of the irq, not the third or fourth. So it is mostly
310 * useful for irq hardware that does not mask cleanly in an
311 * SMP environment.
(gdb) l
312 */
313 for (;;) {
314 spin_unlock(&desc->lock);
315 handle_IRQ_event(irq, ®s, action);
316 spin_lock(&desc->lock);
317
318 if (!(desc->status & IRQ_PENDING))
319 break;
320 desc->status &= ~IRQ_PENDING;
321 }
(gdb) b 315
Breakpoint 5 at 0x100949b7: file irq.c, line 315.
(gdb) c
Continuing.
Next, I'll step into handle_IRQ_event and stop just before entering
the driver.
Breakpoint 5, do_IRQ (irq=2, user_mode=0) at irq.c:315
315 handle_IRQ_event(irq, ®s, action);
(gdb) s
handle_IRQ_event (irq=2, regs=0x10113c40, action=0x50fef380) at irq.c:141
141 irq_enter(cpu, irq);
(gdb) l
136 struct irqaction * action)
137 {
138 int status;
139 int cpu = smp_processor_id();
140
141 irq_enter(cpu, irq);
142
143 status = 1; /* Force the "do bottom halves" bit */
144
145 if (!(action->flags & SA_INTERRUPT))
(gdb) l
146 __sti();
147
148 do {
149 status |= action->flags;
150 action->handler(irq, action->dev_id, regs);
151 action = action->next;
152 } while (action);
153 if (status & SA_SAMPLE_RANDOM)
154 add_interrupt_randomness(irq);
155 __cli();
(gdb) l
156
157 irq_exit(cpu, irq);
158
159 return status;
160 }
161
162 /*
163 * Generic enable/disable code: this just calls
164 * down into the PIC-specific version for the actual
165 * hardware disable after having gotten the irq
(gdb) b 150
Breakpoint 6 at 0x10094813: file irq.c, line 150.
(gdb) c
Continuing.
Breakpoint 6, handle_IRQ_event (irq=2, regs=0x10113c40, action=0x50fef380)
at irq.c:150
150 action->handler(irq, action->dev_id, regs);
So, here we are in the console driver. I think I've made whatever
point I was making, so I'll just delete all the breakpoints, and let
the kernel run so I can log in and halt it.
(gdb) s
con_handler (irq=2, dev=0x10120000, unused=0x10113c40) at stdio_console.c:41
41 stdio_rcv_proc(term->fd);
(gdb)
(gdb) d
Delete all breakpoints? (y or n) y
(gdb) c
Continuing.
Debugging UML in skas mode is pretty much the same as debugging a
normal process, since, in skas mode, UML is pretty normal. You don't
need the 'debug' switch on the command line, and you don't get gdb
popping up in a new xterm. You start by running gdb on the UML binary
and starting it running
host% gdb linux
(host gdb) r
ubd0=/home/jdike/roots/cow.debian,/home/jdike/roots/debian_22
con=port:9000 ssl=pty con0=fd:0,fd:1 umid=debian mem=128M
Also, have gdb ignore SIGSEGV and SUGUSR1 since these are used
internally by UML, and your debugging session would otherwise be
continually interrupted by them
(host gdb) handle SIGSEGV pass nostop noprint
(host gdb) handle SIGUSR1 pass nostop noprint
If you've enabled both CONFIG_MODE_TT and CONFIG_MODE_SKAS, UML will
exec itself, and gdb will see a SIGTRAP.
Program received signal SIGTRAP, Trace/breakpoint trap.
0xa000c000 in _start () at af_packet.c:1903
1903 }
(host gdb)
Just continue it if this happens. If CONFIG_MODE_TT is disabled, then
UML will simply boot up.
If you set any breakpoints before running UML, then all you need to do
is get UML to hit one of them, and you will be bounced back to the gdb
prompt for debugging. When you continue UML, you will be back to
whatever UML prompt you had.
If you need to interrupt UML, you can't ^C it because the terminal is
in raw mode, and the ^C will just hit whatever UML is running. What
you need to do is send the UML kernel thread a SIGINT from another
shell. It is normally the first process after the gdb
uml-2.5 8511: ps ux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
jdike 7132 7.6 2.5 15356 13276 pts/2 S 15:18 0:05 gdb linux
jdike 7133 8.1 3.3 139544 17548 pts/2 R 15:18 0:05 /home/jdike/linux
jdike 7139 5.7 0.1 840 752 pts/2 T 15:19 0:00 [linux]
jdike 7144 0.2 3.3 139544 17548 pts/2 S 15:19 0:00 /home/jdike/linux
jdike 7145 0.0 3.3 139544 17548 pts/2 S 15:19 0:00 /home/jdike/linux
The '[linux]' process is the userspace process, and the kernel process
is the one immediately before it.
host% kill -INT 7133
Debian GNU/Linux 2.2 usermode ttys/0
usermode login:
Program received signal SIGINT, Interrupt.
0xa019a6c1 in __libc_nanosleep ()
(gdb) bt
#0 0xa019a6c1 in __libc_nanosleep ()
#1 0xa00ed092 in idle_sleep (secs=10) at time.c:114
#2 0xa00e5765 in cpu_idle () at process_kern.c:210
#3 0xa000c50e in rest_init () at init/main.c:341
#4 0xa000356d in start_kernel () at init/main.c:435
#5 0xa00f38f7 in start_kernel_proc (unused=0x0) at process_kern.c:153
#6 0xa00e52a2 in run_kernel_thread (fn=0xa00f38d0 <start_kernel_proc>,
arg=0x0, jmp_ptr=0xa020c554) at process.c:239
#7 0xa00f36de in new_thread_handler (sig=10) at process_kern.c:67
#8 <signal handler called>
#9 0xa01891d1 in __kill ()
#10 0xa0188f85 in siglongjmp () at ../sysdeps/generic/longjmp.c:39
#11 0xa00f32e3 in start_idle_thread (stack=0xa020e000,
switch_buf_ptr=0xa020c57c, fork_buf_ptr=0xa020c580) at process.c:275
#12 0xa00f3947 in start_uml_skas () at process_kern.c:169
#13 0xa00ee779 in linux_main (argc=8, argv=0xbffff814) at um_arch.c:376
#14 0xa000c3d2 in main (argc=8, argv=0xbffff814, envp=0xbffff838)
at arch/um/main.c:143
#15 0xa0188d52 in __libc_start_main (main=0xa000c1dc <main>, argc=8,
ubp_av=0xbffff814, init=0xa01c6520 <_init>, fini=0xa01c6cb0 <_fini>,
rtld_fini=0, stack_end=0xbffff80c) at ../sysdeps/generic/libc-start.c:129
(gdb) c
Continuing.
Debian GNU/Linux 2.2 usermode ttys/0
usermode login:
This page describes kernel debugging with UML running in tt mode (go
here for the details on
skas and tt mode). Kernel debugging in skas mode is described
here.
Since the UML runs as a normal Linux process, it is
possible to debug it with gdb almost like any other process. It is
slightly different because the kernel's threads are already being
ptraced for system call interception, so gdb can't ptrace them.
However, a mechanism has been added to work around that problem.
In order to debug the kernel, you need build it from source. See
this page for
information on doing that. Make sure that you enable CONFIG_DEBUGSYM
and CONFIG_PT_PROXY during the config. These will compile the kernel
with -g, and enable the ptrace proxy so that gdb works with UML,
respectively.
Starting the kernel under gdb
You can have the kernel running under the control of gdb from the
beginning by putting 'debug' on the command line. You will get an
xterm with gdb running inside it. The kernel will send some commands
to gdb which will leave it stopped at the beginning of start_kernel.
At this point, you can get things going with 'next', 'step', or 'cont'.
There is a transcript of a debugging session here, with breakpoints being set in the
scheduler and in an interrupt handler.
Examining sleeping processes
Not every bug is evident in the currently running process. Sometimes,
processes hang in the kernel when they shouldn't because they've
deadlocked on a semaphore or something similar. In this case, when
you ^C gdb and get a backtrace, you will see the idle thread, which
isn't very relevant.
What you want is the stack of whatever process is sleeping when it
shouldn't be. You need to figure out which process that is, which is
generally fairly easy. Then you need to get its host process id,
which you can do either by looking at ps on the host or at
task.thread.extern_pid in gdb.
Now what you do is this:
detach from the current thread
(UML gdb) det
attach to the thread you are interested in
(UML gdb) att <host pid>
look at its stack and anything else of interest
(UML gdb) bt
Note that you can't do anything at this point that requires that a
process execute, e.g. calling a function
when you're done looking at that process, reattach to the current thread
and continue it
(UML gdb)
att 1
(UML gdb)
c
Here, specifying any pid which is not the process id of a UML thread
will cause gdb to reattach to the current thread. I commonly use 1,
but any other invalid pid would work.
Running ddd on UML
ddd works on UML, but requires a special kludge. The process goes
like this:
Start ddd
host% ddd linux
With ps, get the pid of the gdb that ddd started. You can ask the gdb
to tell you, but for some reason that confuses things and causes a hang.
run UML with 'debug=parent gdb-pid=<pid>' added to the command line
- it will just sit there after you hit return
type 'att 1' to the ddd gdb and you will see something like
0xa013dc51 in __kill ()
(gdb)
At this point, type 'c', UML will boot up, and you can use ddd just as
you do on any other process.
Debugging modules
gdb has support for debugging code which is dynamically loaded into
the process. This support is what is needed to debug kernel modules
under UML.
Using that support is somewhat complicated. You have to tell gdb what
object file you just loaded into UML and where in memory it is. Then,
it can read the symbol table, and figure out where all the symbols are
from the load address that you provided. It gets more interesting
when you load the module again (i.e. after an rmmod). You have to
tell gdb to forget about all its symbols, including the main UML ones
for some reason, then load then all back in again.
There's an easy way and a hard way to do this. The easy way is to use
the umlgdb expect script written by Chandan Kudige. It basically
automates the process for you.
First, you must tell it where your modules are. There is a list in
the script that looks like this:
set MODULE_PATHS {
"fat" "/usr/src/uml/linux-2.4.18/fs/fat/fat.o"
"isofs" "/usr/src/uml/linux-2.4.18/fs/isofs/isofs.o"
"minix" "/usr/src/uml/linux-2.4.18/fs/minix/minix.o"
}
You change that to list the names and paths of the modules that you
are going to debug. Then you run it from the toplevel directory of
your UML pool and it basically tells you what to do:
******** GDB pid is 21903 ********
Start UML as: ./linux <kernel switches> debug gdb-pid=21903
GNU gdb 5.0rh-5 Red Hat Linux 7.1
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) b sys_init_module
Breakpoint 1 at 0xa0011923: file module.c, line 349.
(gdb) att 1
After you run UML and it sits there doing nothing, you hit return at
the 'att 1' and continue it:
Attaching to program: /home/jdike/linux/2.4/um/./linux, process 1
0xa00f4221 in __kill ()
(UML gdb) c
Continuing.
At this point, you debug normally. When you insmod something, the
expect magic will kick in and you'll see something like:
*** Module hostfs loaded ***
Breakpoint 1, sys_init_module (name_user=0x805abb0 "hostfs",
mod_user=0x8070e00) at module.c:349
349 char *name, *n_name, *name_tmp = NULL;
(UML gdb) finish
Run till exit from #0 sys_init_module (name_user=0x805abb0 "hostfs",
mod_user=0x8070e00) at module.c:349
0xa00e2e23 in execute_syscall (r=0xa8140284) at syscall_kern.c:411
411 else res = EXECUTE_SYSCALL(syscall, regs);
Value returned is $1 = 0
(UML gdb)
p/x (int)module_list + module_list->size_of_struct
$2 = 0xa9021054
(UML gdb) symbol-file ./linux
Load new symbol table from "./linux"? (y or n) y
Reading symbols from ./linux...
done.
(UML gdb)
add-symbol-file /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o 0xa9021054
add symbol table from file "/home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o" at
.text_addr = 0xa9021054
(y or n) y
Reading symbols from /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o...
done.
(UML gdb) p *module_list
$1 = {size_of_struct = 84, next = 0xa0178720, name = 0xa9022de0 "hostfs",
size = 9016, uc = {usecount = {counter = 0}, pad = 0}, flags = 1,
nsyms = 57, ndeps = 0, syms = 0xa9023170, deps = 0x0, refs = 0x0,
init = 0xa90221f0 <init_hostfs>, cleanup = 0xa902222c <exit_hostfs>,
ex_table_start = 0x0, ex_table_end = 0x0, persist_start = 0x0,
persist_end = 0x0, can_unload = 0, runsize = 0, kallsyms_start = 0x0,
kallsyms_end = 0x0,
archdata_start = 0x1b855 <Address 0x1b855 out of bounds>,
archdata_end = 0xe5890000 <Address 0xe5890000 out of bounds>,
kernel_data = 0xf689c35d <Address 0xf689c35d out of bounds>}
>> Finished loading symbols for hostfs ...
That's the easy way. It's highly recommended. The umlgdb script is
available in the UML
utilities tarball in tools/umlgdb/umlgdb. The
hard way is described below in case you're interested in what's going
on.
Boot the kernel under the debugger and load the module with insmod or
modprobe. With gdb, do:
(UML gdb) p module_list
This is a list of modules that have been loaded into the kernel, with
the most recently loaded module first. Normally, the module you want
is at module_list. If it's not, walk down the next links, looking at
the name fields until find the module you want to debug. Take the
address of that structure, and add module.size_of_struct (which in
2.4.10 kernels is 96 (0x60)) to it. Gdb can make this hard addition for
you :-):
The offset from the module start occasionally changes (before 2.4.0, it
was module.size_of_struct + 4), so it's a good idea to check the init and
cleanup addresses once in a while, as describe below.
Now do:
Tell gdb you really want to do it, and you're in business.
If there's any doubt that you got the offset right, like breakpoints
appear not to work, or they're appearing in the wrong place, you can
check it by looking at the module structure. The init and cleanup
fields should look like:
with no offsets on the symbol names. If the names are right, but they
are offset, then the offset tells you how much you need to add to the
address you gave to add-symbol-file.
When you want to load in a new version of the module, you need to get
gdb to forget about the old one. The only way I've found to do that
is to tell gdb to forget about all symbols that it knows about:
(UML gdb) symbol-file
Then reload the symbols from the kernel binary:
(UML gdb) symbol-file /path/to/kernel
and repeat the process above. You'll also need to re-enable breakpoints.
They were disabled when you dumped all the symbols because gdb
couldn't figure out where they should go.
Attaching gdb to the kernel
If you don't have the kernel running under gdb, you can attach gdb to
it later by sending the tracing thread a SIGUSR1. The first line of
the console output identifies its pid:
tracing thread pid = 20093
When you send it the signal:
host% kill -USR1 20093
you will get an xterm with gdb running in it.
If you have the mconsole compiled into UML, then the mconsole client
can be used to start gdb:
(mconsole) (mconsole) config gdb=xterm
will fire up an xterm with gdb running in it.
Using alternate debuggers
UML has support for attaching to an already running debugger rather than
starting gdb itself. This is present in CVS as of
17 Apr 2001. I sent it to Alan for inclusion in
the ac tree, and it will be in my 2.4.4 release.
This is useful when gdb is a subprocess of some UI, such as emacs or ddd. It
can also be used to run debuggers other than gdb on UML. Below is an example
of using strace as an alternate debugger.
To do this, you need to get the pid of the debugger and pass it in with the
'gdb-pid=<pid>' switch along with the 'debug' switch.
If you are using gdb under some UI, then tell it to 'att 1', and you'll find
yourself attached to UML.
If you are using something other than gdb as your debugger, then you'll need
to get it to do the equivalent of 'att 1' if it doesn't do it automatically.
An example of an alternate debugger is strace. You can strace the actual
kernel as follows:
Run the following in a shell
host%
sh -c 'echo pid=$$; echo -n hit return; read x; exec strace -p 1 -o strace.out'
Run UML with 'debug' and 'gdb-pid=<pid>' with the pid printed out by the
previous command
Hit return in the shell, and UML will start running, and strace output will
start accumulating in the output file.
Note that this is different from running
host% strace ./linux
That will strace only the main UML thread, the tracing thread, which doesn't
do any of the actual kernel work. It just oversees the virtual machine. In
contrast, using strace as described above will show you the low-level activity
of the virtual machine.
When you start UML, it will create a file whose size is the same as
the UML physical memory size, which is whatever you specified with the
'mem=' switch or its default. Initially, none of that is backed with
real host memory, but that changes as it touches its physical memory.
Its host memory consumption will asymptotically approach its physical
memory limit as it reads data into cache. That consumption will not
tend to decrease for two reasons
UML, like any other Linux kernel, will keep data in cache as long as
there is no pressing need for memory. This makes sense for a native
kernel running on hardware since there is no other possible use for
the memory. However, for UML, the host quite possibly could make
better use of any under-used memory.
Even if UML did release a bunch of cache, it has no way of
communicating that to the host, which will still see that memory as
being dirty, and needing preservation, by swapping it if necessary.
Another problem is the unnecessary duplication of data in host
memory. If you have a number, say 'n', of UMLs booted from similar
disk images, the data that's the same is present 2 * n times in the
host memory, n times in the host page cache, and n times in the UML
page caches.
Booting those UMLs from the same image with COW files will reduce the
number of copies in the host page cache to 1, but there will still be
n copies in the UML page caches, one in each UML. These copies can be
eliminated with the use of ubd=mmap, which causes the ubd driver to
mmap pages from disk images rather than using read() and write().
This causes UML to share page cache with the host.
In order to see any reduction in host memory use, it is necessary to
free the UML phsyical memory pages that were over-mapped by the ubd
driver. If these are mapped in from tmpfs or any other filesystem,
they won't be freed since filesystems preserve data even when the data
isn't mapped in anywhere. So, this is where the host /dev/anon driver
comes in. It has the property that when a page is mapped in, and
subsequently unmapped, the page will be freed back to the host, and
any data that was stored there will be lost. Obviously, this is wrong
for a filesystem, but this is exactly what's needed for UML.
/dev/anon
To have UML use /dev/anon, you need to do the following:
apply the devanon patch (available from the
download
page) to your host kernel
make /dev/anon a character device with major number 1 and minor 10
mknod /dev/anon c 1 10
with read-write permissions for everyone (or at least everyone who
will be running UML)
chmod 666 /dev/anon
get a UML that supports /dev/anon (2.4.23-2 or later)
run it with the 'ubd=mmap' switch
make sure that the UML filesystems have 4K (one page) blocksize
NOTE: At this point, 'ubd=mmap' is known to eat filesystems, so
don't try this yet with data that you care about. If you use COW
files, the backing files are safe since they're opened read-only, but
any data that UML has write access to is at risk from 'ubd=mmap'.
Use of the 'ubd=mmap' switch is needed in order to get any memory use
benefits from /dev/anon. However, UML will use it regardless of
whether the ubd driver is doing mmap. This is almost the same as
having it use files in /tmp for its physical memory, except that the
UMLs won't be limited to the capacity of /tmp. This makes the host
management somewhat easier. Without /dev/anon, you need tmpfs mounted
on /tmp for optimal UML performance. With it, you get the same
performance without needing to make sure that your tmpfs mount is big
enough for the UMLs running on the system.
Memory savings
In my testing, which involves booting a number of instances on a
Debian image to a login prompt, the combination of /dev/anon and
'ubd=mmap' results in about a 25% decrease in host memory
consumption. This was measured by looking at the decrease in free
memory per UML when the host was not under memory pressure. Without
'ubd=mmap', this is ~28M. With 'ubd=mmap', this went down to ~21M. I
checked this by counting the number of instances I could boot before
the host went into swap. This increased from 16 to 20, again a 25%
increase.
This is obviously workload-dependent. Workloads that involve lots of
readonly data, such as code, will benefit more than those that involve
read-write data.
Performance of 'ubd=mmap'
People commonly ask if switching the ubd driver from using read() and
write() (and the data copying they do) to using mmap (and the tlb
flushing that it does) will help the performance of a single UML. The
answer is that the performance is probably about the same either way.
The reason is that mmap is an expensive operation for the CPU. People
have measured how large data copies have to be in order to be more
expensive than playing games with mmap.
The size that I've seen is about 8K, or two pages on x86. Since UML
is doing maps with 1 page granularity, which is in the ballpark, you'd
expect the performance to be about the same. My testing, which has
mostly been kernel builds, bears this out. The speed is pretty close,
to within the error, either way.
Future work
This is leading to more flexibility in managing the memory consumption
of UMLs and the host. The next step is to allow the host to add and
remove memory from UMLs at run-time, i.e. hot-plug memory.
This will allow a utility on the host to keep track of the memory
usage of the UMLs and the host, and shuffle memory between the UMLs
and to and from the host in order to implement whatever policy is
desired.
I am planning a shipping a sample utility which implements the
host-should-not-swap policy. This would keep track of the free memory
on the host, and, when it gets low, it would remove memory from an
idle UML and release it to the host. When the host has ample memory,
it would look for a UML that's under memory pressure and give it some
memory.
It is also possible to use this facility to reverse the monotonic
increase in UML memory consumption described above. By pulling memory
out of a UML until it squeals, you will force it to drop its page
cache, resetting its host memory consumption to what it was much
earlier in its life. Doing this periodically to all the UMLs on the
host will result in their cached data being much more up-to-date, at
the expense of some performance loss from old cache data not being
available and needing to be reread.
These are probably a decent start for any UML site, and might be
sufficient for many, but some may need something different. For
example, a UML ISP might have different classes of UMLs on the same
host, and would want the expensive UMLs to have preference over the
cheap ones for whatever memory is available. UML will provide
mechanism, plus an example policy or two, and it will be up to the
user community to implement whatever policies are appropriate for host
memory management.
This page contains information about what's currently happening with the
project. I may update it once in a while if I feel like it.
26 Apr 2006
The time namespace patches I sent out just before leaving for Brazil
got a bunch of comments pointing out things I did wrong. The
PTRACE_SYSCALL_MASK interface is wrong because strace -e (which is the
other possible user) has opposite requirements from UML. It wants to
selectively trace system calls rather than selectively not trace
them. There were also some bugs in the implementation.
Eric Biederman pointed out some failings in the actual time
virtualization code. What I did won't work so well when you want to
migrate a container from one host to another. This will require some
thought.
LWN picked up on it and Jon Corbett wrote a better summary
of these patches than I managed.
I spent last week in Porto Alegre, Brazil for FISL 7.0.
It's a pretty good show, thought somewhat less technical than the
likes of OLS and LCA. It's larger, with ~4000 attendees registered,
and it had a small exhibition floor. There were a good number of
international speakers, and we all seemed to be in one track, in the
largest room, with simultaneous translation into English, Spanish, and
Portuguese. My own talk was somewhat sparsely attended. The
user-level, eye-candy-type talks were much better attended.
The UML Book is now out! There was a bit of a delay at the printer,
but it is now available, and people have it in their hot little
hands. It's available from Prentice-Hall
and from Amazon.
13 Apr 2006
I've been slowly redoing the UML web site. The idea is to make it
more friendly to newbies, by having more step-by-step instructions for
the normal ways of doing things and less reference-type dumps of
information. The reference stuff will still be there, but it won't be
the first thing you see. What I have so far can be seen
here.
I implemented time namespaces in the host and support for them in
UML. The idea is to virtualize time in the host by creating a
partition which has its own independent clock. This partition takes
the form of a namespace which is created by the new unshare system
call. Within this partition, a settimeofday call will just change the
offset inside the namespace without changing the system time.
gettimeofday reads the system time and adds the offset.
The advantage to UML is that this allows gettimeofday to run on the
host as though it were running inside UML. The results are the same,
except that the system call doesn't need to be traced. In order to
make this work, I also had to add a mechanism for system call tracing
to be selective - I want to turn off tracing of gettimeofday while
retaining interception of everything else. This is done with the new
PTRACE_SYSCALL_MASK, which takes a bitmask saying which system calls
will be traced and which won't.
With this stuff working, you'd expect that gettimeofday would be
pretty close to native speed, since it runs on the host without UML
doing anything. The two measurements I've done, with a loop of a
million calls, are 98.8% and 99.2% of native.
I need to get the host side of settimeofday working, and then I will
send this off to LKML as an RFC.
31 Mar 2006
All of the patches that BB and I sent to akpm are now on their way to
mainline. So, 2.6.17 will have TLS and hotplug memory support. The
TLS support, in particular, was a long time coming, but it works very
well and has received a lot of testing, so I don't expect many
problems. It would be too optimistic to expect no problems when
exposing something new to a much larger user base, but I don't expect
any problems to be very large.
OLS papers are due tomorrow (April 1), so I've been working on
mine for the last couple of days. Eric Beiderman is giving a
talk on extending namespaces to cover the entire kernel, so there's
going to be some overlap there, since that's also relevant to my
hypervisor talk.
28 Mar 2006
A bunch of UML patches have gone to Andrew in preparation for 2.6.17.
I sent in 10 more today, which were mostly Al Viro's UML cleanup
patches. The one that was mine was a cleanup of the earlier printf
patch.
BB sent in his TLS patches. They've been tested for a long time, and
have no known bugs - the last one was spotted and fixed a few days
ago.
This eliminates a whole lot of patches from my development tree. With
-mm2, I should be into the low 40s. There will be three major
(i.e. multipatch) projects left to be merged
externfs, hostfs, and humfs - humfs can't currently do a kernel build,
and all of that code is in one directory, where it should probably be
three. Also, we have questions about the structure of the underlying
code, mainly the stuff that deals with filehandles.
UML/S390 - Most of the preparatory patches are merged. The signal
hander restructuring is still sitting there, as are a couple of
patches which I really don't like. The add-gate-vmas patch works, but
it's fairly gross. The fix-jiffies patch is small but it seems like
the wrong way to go about it. Also, there has been some bit-rotting,
as the skas0 interfaces have changed since I merged Bodo's patches
into my tree.
The ubd driver rework - I have a report, which I can't reproduce, that
this hangs under heavy I/O. Also, I have yet to sort out dealing with
the early I/O on COW files with O_DIRECT in the picture.
In other news, the sanitized kernel headers project was restarted.
This is of interest because that can, if done right, greatly simplify
UML's use of the host arch's headers. Now, most of them can just be
reused by UML. However, some of them are mostly usable, but have
stuff which is wrong for UML. I either copy these into asm-um,
leaving out the objectionable bits or include them, but use various
nasty tricks to get rid of the parts I don't want. Usually, this
means using defines before including them to rename the things I don't
want.
With a clean set of kernel headers, that won't be necessary any more,
assuming that they include all of the userspace-usable things, rather
than just the things that make up the kernel ABI. Kyle Moffett has
seemed amenable to accommodating UML, so we might get some nice UML
header cleanups from his KABI work.
25 Mar 2006
I fixed the humfs hang, so I consider it to be stable at this point.
However, I haven't yet tried a kernel build loop on it.
BB sent out his current TLS patchset for comment and review. I
dropped it into my akpm tree, which contains the patches destined for
-mm. It works fine, and fixes a problem that I had been chasing. So,
it looks good for 2.6.17, except that the patchset itself is a bit
disorganized.
This morning, I sent out 16 patches to Andrew for routing to
mainline. They include
the rest of Gennady Sharapov's isolation of libc code
fixing the get_user warnings that popped up on current gcc
memory hotplug
allowing a ubd device to be shared among clustered UMLs
a handful of smallish bug fixes
22 Mar 2006
With hostfs mostly out of the way, I started looking at humfs. This
was fairly easy. The bugs were mostly due to bit rot, and easily
fixed. The one exception was a bug in the symlink handling in
humfsify. It turns out that both -d and -l are true in Perl for a
symlink that points to a directory. This was faking humfsify into
treating such symlinks as directories.
On an intensive I/O load on humfs, I get hangs after a while. The AIO
thread is somehow faked into calling io_getevents when there are none
to get. My investigation into this uncovered some weaknesses in AIO
handling of -EAGAIN, when no requests can be queued until some have
been retrieved with io_getevents. The ubd driver assumes that, if it
gets -EAGAIN from an aio submission, that some pending requests are
ubd requests, and the queue will be restarted from the ubd interrupt
handler.
However, if the AIO queue is filled with humfs requests, there will be
no ubd request to wake up the queue, so the ubd driver will stall.
This isn't hard to fix, but there are some subtleties, mostly in
avoiding races.
19 Mar 2006
I found a couple of inode refcounting bugs in externfs. When these
were fixed, I can do kernel builds on hostfs. What was happening was
that I was giving inodes extra references. When the files were
deleted, they were deleted on the host, but the inodes within UML
remained. At some point, a new file on the host would get the same
inode number as the deleted file. When externfs looked up this inode
number, it got the inode structure belonging to the old file since
that was never thrown out due to the extra reference counts. This let
to directories looking like files (when the old inode was a file) and
to normal files having the wrong length (because the length was taken
from the old file).
There are some things which still don't work. Loop-mounting an image
from a hostfs filesystem doesn't work (and never did, even with the
old hostfs) because it doesn't support sendfile. I added that, and it
turns out that the commit_write method is broken in a few ways. It
returns a byte count, rather than 0/-errno, which is wrong. It also
supports only mmap, rather than read/write to the host.
17 Mar 2006
I spent this week banging on the new, externfs-based hostfs. The book
is going to be out in a few weeks, and there are some things it
promises which I need to get working. The new hostfs and humfs are
among them. I've killed a bunch of bugs, and am currently chasing one
more. Everything I've found has been in the externfs layer, so I'm
debugging humfs at the same time. Hopefully that will work without
too much trouble once I'm happy with externfs.
8 Mar 2006
I implemented open, read, and close for umlfs, so now you can actually
look at the files you've exported from your UML to the host. I spent
most of the day tracking down a stupid vfsmount and dentry refcounting
bug. Once that was fixed, umlfs started working nicely again.
I found and fixed a bug in my FUSE async support. It turns out I was
referencing something right after it was freed.
1 Mar 2006
FUSE will fully support async operation when Miklos sends my patches
to mainline. I did a second round of the O_ASYNC patch and another
patch to enable O_NONBLOCK. Miklos queued them both, so we'll
probably be seeing them in mainline.
I'm also finding inconveniences in the FUSE library, which I am
complaining about and sending patches for, starting with a receive
routine which doesn't let me know if the /dev/fuse read returned
-EAGAIN. Miklos is being receptive, and fixing things.
In actual UML work, I implemented readlink today, so you can cd and ls
around inside the UML filesystem on the host without many problems.
There are still some problems on filesystem boundaries which I don't
understand yet.
BB made a release of the uml_utilities which contains a good number of
cleanups, including some 64-bit fixes in the COW file utilities. The
header wasn't specified correctly for 64-bit boxes, with the result
that a 64-bit system would misread a COW header produced by a 32-bit
system. I merged his changes into my tree, which contains the umlfs
utility, plus some mconsole work. I'll be making an official release
once I get some data off a laptop disk. Among that data is the
Makefile which I use to actually upload new releases of the utilities
and other UML-related files.
27 Feb 2006
Miklos (the FUSE guy) announced mountlo today, which is very similar
to the FUSE thing that I've been doing. He wrote this for a very
different purpose - to allow user-level mounting of host filesystems,
where UML is just an enabling tool. His design is also very different
- he creates a device inside UML and has a UML process doing the file
operations. In related news, I polished the fuse-async patch a little
and sent it to Miklos.
I finished my review of the page proofs of the UML book, fixing a
number of errors, some of which were pretty embarrassing. I think
this is the last thing I have to do with it until it's published. At
this point, it's on automatic pilot inside the publisher.
24 Feb 2006
I got the umlfs FUSE filesystem working a little. lookup and readdir
now work, so you can now cd and ls in a UML filesystem on the host.
There is a dentry corruption problem, which causes crashes later (on
shutdown usually). I think I'm either double-freeing a dentry somehow
or dputting it too often. Today I looked at adding asynchronous
notification to FUSE, so that the /dev/fuse file descriptor can
generate a SIGIO when there's something the userspace server has to
do. Lack of this is annoying since I have to generate other types of
interrupts to UML in order to get it to service FUSE requests.
Normally, this means whacking the keyboard. I am building the
O_ASYNC-enabled FUSE now, and we'll see how well it does.
20 Feb 2006
I started seriously working on putting a FUSE server inside UML to
export the UML filesystem to the host. It was easier than I expected,
but needed some effort because the FUSE library wants to do things
during initialization that can't easily be done inside UML, such as
setting signals, forking, and execing. What I ended up doing is
putting that stuff in a separate helper process, which gets a file
descriptor to /dev/fuse and passes that to UML through mconsole.
Inside UML, the driver does the rest of the initialization, like
creating a fuse session and registering an operations vector.
I copied the "hello" filesystem into my driver for testing purposes,
and that mostly works. One wart is that /dev/fuse doesn't support
SIGIO, so UML can't easily get an interrupt when a FUSE operation
needs to be handled. I'm planning on fixing this and sending the
patch to Miklos.
The UML book is nearing print. I'm currently reviewing a PDF
containing page proofs. This gives the thing a reality that it didn't
have before. Amazon
knows about it already, and it is currently 2,382,397 in sales
rank. The official release date is April 7.
8 Feb 2006
There is a virtualization infrastructure thread happening on LKML
right now which looks like it fits nicely with my thinking on having
UML do less system call tracing. The OpenVZ and CKRM folks are
interested in this, as is Eric Biederman, who wants to migrate
workloads around a compute cluster. The thinking is to introduce
namespaces for the various resources that the kernel controls and make
compartments by creating new namespaces and grouping them together.
For UML, this would mean creating a compartment which contains enough
UML data that system calls execute directly on the host, and do the
right thing.
Two of the eight patches I sent to mainline got dinged yesterday.
Ulrich Drepper didn't like my resurrection of the internal jmp_buf
defines that he got rid of, and suggested that I just reimplement
setjmp and longjmp myself. Linus didn't like the uaccess warning
patch since I apparently got rid of the warnings by making changes to
the declarations which didn't make any sense.
2.6.16 seems to be winding down now, so I think I'm through with
pushing patches into it.
6 Feb 2006
Last week, I discovered that rc1-mm4 didn't boot on x86_64, while
everything up to -mm3 did. A UML process would die strangely after
having a page fault handled. This is an invitation to do a bisection
search on the patches between the two releases in order to find the
culprit patch. I just did the search on the UML patches rather than
everything, and that turned up a patch which did nothing but move code
from one file to another. Further poking at that patch revealed that
moving a jmp_buf from one file to another was enough to cause the
crash.
Needless to say, this was uninformative. There was another clue, in
the form of a complaint by the host kernel about a bogus signal frame
whenever UML crashed like this. I pursued this by comparing "good"
signal frames to "bad" ones, and decided that the floating-point
register values must be to blame. It turns out that these weren't
initialized to any sort of sane values when a new UML process was
created. When I fixed this, the crash went away. I still have no
idea what the connection is with the jmp_buf moving around.
Last week, Andrew released rc1-mm4 and Linus released -rc2. After
tracking down that bug this morning, I updated my trees, throwing out
patches that had made it into mainline. After this, I had 67 patches
in both trees. Then, I sent 8 more to Andrew. These are small fixes
and cleanups that had accumulated over the last couple of weeks.
30 Jan 2006
With FUSE now in mainline, I started thinking about umlfs, which would
export, through FUSE, a UML filesystem to the host. I think this
would provide pretty much everything that the mconsole exec fans
want. If you chroot yourself to this mount, you will be running the
UML binaries, looking at the UML /proc, etc. So, you'd be able to fix
passwords and look at running processes. The things that wouldn't
work as expected would be things that don't go through the UML
filesystem. For example, ifconfig creates a socket and calls ioctl on
it to get interface information. This is a host socket, so you'd just
be looking at the host's interfaces. Anything that looks at the UML
filesystem will report on the UML.
This started me thinking. We would have processes that are acting
almost exactly like they are running inside a UML, except they are not
being system call traced like normal UML processes. Instead, they
are coupled to the UML at the subsystem, i.e. the filesystem layer,
level. This provides a whole new way of thinking about how to do
virtualization.
This is essentially migrating a UML process to the host. When you
migrate a process from one system to another, the process can't know
that it has moved. The new system has to proxy the old system's
information to the process. This can be done by redirecting every
system call back to the home system, which is how UML works. However,
that's not necessary. When you do file accesses, there are perfectly
good network filesystems which will readahead and cache the remote
system's data so that a process doesn't have to call back to its home
system on every file operation. This is what the FUSE does (or could
do) with the umlfs data. File operations would run at full speed as
long as FUSE has the required UML data cached already.
The question this raises is whether it's possible to do this in
general - have nearly all UML process system calls run untraced on the
host, with the host proxying the UML data such that those system calls
run at full speed. This would have to include
memory - all cached data would have to be in memory owned by UML in
order to maintain mmeory jailing
processes and networking - the host would have to show UML processes
the UML network and the other UML processes. This information would
need to be passed from the UML to the host and kept up to date there
in order to minimize callbacks to the UML.
kernel version skew would be an issue - a process using a new flag
that the host didn't implement would get a -EINVAL if the system call
runs natively on the host. So, there would need to be some selective
system call tracing when this is a problem.
So, this would require some significant support in the host kernel in
order to run, but raises the possibility that UML at some point could
mostly do away with system call tracing and run at nearly native
speed. How close it gets is determined by how often the host would
have to make UML processes wait for information from UML.
Since this is really implementing process migration, this will be of
interest outside the virtualization world. The same support would be
usable for migrating processes between systems, except that
communication between those systems would be over the network rather
than over memory as with a host and a UML running on it.
And once you have process migration, you start to think about
full-blown clustering. With migration, you are mirroring one system's
data on another. Clustering takes that one step further and does away
with the idea that data belongs to a particular system. The data
belongs to the cluster and lives wherever it is most convenient.
So, implementing virtualization as process migration makes a
not-quite-complete migration system useful as a performance boost for
UML. Finishing it by implementing the network support needed for two
physical systems to migrate processes would provide full process
migration to Linux. And that will get people thinking about
implementing full SSI clustering.
27 Jan 2006
I have a bunch of patches in my queue which together redo how the ubd
driver works. I've been wanting to send it to Andrew for a while,
except that they broke COW. This week, I tracked down the problem. I
wasn't specifying the length of bitmap I/O requests properly, causing
some parts of the COW bitmap not to be written out. The next time the
filesystem was mounted, these missing parts of the bitmap would cause
data to be read from the backing file rather than the COW file,
causing file corruption.
With that fixed, I upgraded my FC4 filesystem to FC5 Test, and it
seems to be working pretty well. The most obvious difference is that
udev is a lot faster in FC5.
18 Jan 2006
-mm4 broke the UML build in a couple of ways. There is a new thread
flag, TIF_RESTORE_SIGMASK, which, among other things, allows some
signal code to be pulled from the arches and made generic. UML
didn't define this, or support it, so the build broke. Adding
support was fairly simple, so I did it.
There was a __iowrite32_copy function introduced which is built in to the
kernel unconditionally, even though UML is most unlikely to use it.
It calls __raw_writel, which UML didn't define. I stole the
definition for this (and the "b" and "w" variants while I was at it)
from x86.
My asm-offsets patch had a bug in it which broke the build. This
fixed, but not yet on its way to Andrew.
After these, UML builds and runs again.
The soft interrupts patch is now in -mm, finally. It hasn't been sent
to Linus yet, but I told Andrew that it is 2.6.16 material.
Linus released 2.6.16-rc1 and Andrew did -mm1 in quick succession. I
duly updated my trees, throwing out a number of patches (which had
been merged) in the process. My -mm tree has 58 patches in it, which
is a nice decrease from the ~90 I had before. When I push out the
ubd cleanup and the TLS stuff, that looks like it will go under 40,
which is much more manageable.
BB took back his TLS patches which I had cleaned up some, and cleaned
them up some more, so that they build on all combinations of
CONFIG_MODE_TT and CONFIG_MODE_SKAS. I dropped those back in my
tree, replacing my old ones. They seem to work OK, except that his
system seems to define a struct user_desc and mine doesn't. I have a
struct modify_ldt_t_s instead. We'll need to sort out which one we
can count on. I also deleted an annoying error message. These need
some more looking at, but they're closer to being sent to Andrew.
8 Jan 2006
The aforementioned half-dozen patches are now on their way to Andrew.
The nasty compilation fix is now a lot nicer. It turns out that I
could use the existing asm-offsets.h mechanism, which fixed the
problem and let me delete some Makefile crud.
All of my previous patches are now in the hands of Linus, so they
should be in his tree soon, if they're not there already.
6 Jan 2006
I sent Andrew a smallish set of four patches - three libc code
isolation patches and the futex.h consolidation. The futex.h patch
came about because I needed to revert the UML futex.h back to the
original version, which just returned -ENOSYS. It turns out that most
of the architectures had their own identical copies of this. Rather
than have UML create yet another copy, I stuck it in asm-generic and
made the other arches use that.
I spent today cleaning up the debris from the earlier patches that
came out in -mm1. It turned out that the boot output was printed
twice due to the mconsole stack patch. This isn't happening in my
main 2.6.15 tree for some reason, so I didn't notice until I saw it
with -mm1. There was also a nasty compilation problem which I have a
nasty fix for, plus reverting my earlier patch that checks at
compile-time that either MODE_TT or MODE_SKAS is enabled. Adrian Bunk
sent in a Kconfig way of doing it.
Beyond that, I have about a dozen more patches ready to go. These are
the ones that lead to softints. This has been in my tree forever, and
it speeds up UML quite nicely, so it's about time it went to mainline.
With these in mainline, my own patchset goes from the low 90s down to
the 50s, which is a much more manageable number. To get more patches
out of my tree, I'm thinking to clean up the ubd driver series and the
TLS/NPTL series and send them to Andrew for inclusion in 2.6.17.
That's about 15 more. If that happens, I will be down to around 40
patches, which is less than half of what I started with.
3 Jan 2006
Linus released 2.6.15 yesterday. It contained a last-minute set of
fixes from Paolo which fixed segfaults caused by running printk on the
wrong stack, cleaned up some code, and cleaned up the compilation. I
pulled it, and it seems healthy.
That release was my cue to send in a bunch of patches to Andrew. I
sent twelve, which were the bunch that I did in the last couple of
weeks, cleaning up the console code, and a bunch of other cleanups
that I did along the way.
Next on the list is the umid OS abstraction and the mconsole printk
interception that I did last week. Then, I'm looking at the ubd patch
series that I've had for a while.
31 Dec 2005
I generalized the mconsole printk thing a bit, and made sysrq use it.
Now, when you invoke sysrq, you get any output back in the mconsole
client. This is especially nice with sysrq t since that gives you the
stack of every process on the system. It's much better to get it from
the mconsole client than to have to go looking through the UML's dmesg
in order to see it.
29 Dec 2005
While playing with the UML consoles last week, I noticed that it is
possible to register consoles at runtime, and this makes it possible
to send printk output back to an mconsole client. This is done by
registering a console from the mconsole driver. This console is
called whenever there is output. Normally, it ignores the output and
just returns. However, when a "stack" command is active, it collects
the output and sends it to the client. This will capture the stack
and registers nicely. It will also capture any other printk that
happen to occur at the same time. I don't know of a way of ignoring
any such output, but that shouldn't happen very often.
Now, I need to hook sysrq up to this mechanism because it has had the
same problem. It is also possible to monitor all printk output from
the host. I would do this by adding another mconsole notification for
the kernel log. Whenever there is printk output, it would be sent to
the mconsole notification socket and whatever is listening to it.
In doing this, I learned something about Unix sockets. There is a
limited number of packets that can be pending at once, and it is easy
to fill it up. I was trying to figure out why the mconsole client
printed only a couple of lines of stack and then hung. It turns out
that the sendto from the mconsole driver was returning -EAGAIN and I
wasn't checking for it. This happened after only a few hundred
characters of output, which seemed a bit thin. After looking at the
code, it turns out that there is a limit of 10 (by default - this is
tunabled from /proc or sysctl) packets pending on a Unix socket. Send
another and you get -EAGAIN.
I was sending one packet per printk call, some of which were pretty
small. I changed things so that the output gets accumulated into a
buffer, which is sent out when it fills up. This is more complicated,
but it makes things work a lot better.
25 Dec 2005
Last week was console week. I decided to figure out what was causing
the -EBADFs in deactivate_all_fds on shutdown sometimes. This is a
consequence of closing a file descriptor and forgetting to shut down
the associated IRQ. In the console code, it is not obvious that when
the close happens, the IRQ is also freed. So, I decided restructure
the code to make it so. In doing so, I discovered a bunch of other
things that needed fixing and cleaning up.
I ended up with 10 new patches, which do things ranging from code
reformatting with no functional changes to fixing the console behavoir
when pasting a large amount of data into one.
That last problem has been a long-standing one. It turned out to be
two problems. One is that the console driver never implemented
throttling, which is how the tty driver tells the hardware driver to
stop sending it data. Adding this caused large pastes to stop losing
data.
The other problem was that the process receiving the large paste would
see an EOF in the middle and exit, and the shell would receive the
rest. This turned out to be caused by a bug in the driver exercising
a bug in the tty driver. The tty driver bug was that it was update a
counter before queuing a character, assuming that the enqueuing would
succeed. However, it can fail, and the counter being updated for a
non-existant character can cause premature EOFs to be emitted by the
tty driver.
The UML bug was that, while it detected a full tty buffer and
rescheduled the processing of the current interrupt, it delayed it
only long enough to return from the IRQ handler, because the tasklet
was scheduled immediately. Delaying that for one jiffy fixed that
bug.
14 Dec 2005
There has been a problem with the TLS patches since they've been in my
patchset - they don't
build on x86_64. This is now fixed. As part of my cleanup of the TLS
stuff, x86_64 now builds and runs with the full patchset. I did
implement a bunch of do-nothing stubs for x86_64, which is worrisome
since it should have the same requirements as i386. However, my
x86_64 filesystem boots fine, and always has. This will need looking
into later.
9 Dec 2005
The Book has pretty much been handed over to the publisher now. I
finished some last-minute items like the acknowledgements, bio, and
artwork earlier this week. I still need to reread the thing to see if
there's anything that needs fixing, but aside from that, it is done.
Another book showed up on
Linux Journal. It's not really entirely about UML - it's about
debugging and performance tuning on Linux - but there's a chapter on
UML, Linux Journal chose that chapter to excerpt (or the publisher
chose it to make available for excerpting).
On actual UML work, Blaisorblade's NPTL patches broke in an interesting way
with 2.6.15--rc5-mm1. I started getting lots of complaints
about PTRACE_SET_THREAD_AREA calls failing, and failing with a bogus
errno. After looking at this, it turns out that those calls had
always been failing, but failing silently because
ptrace was called from kernel code
in kernel files, errno is renamed to kernel_errno to avoid conflicting
with the libc errno
so uses of errno in kernel files will refer to kernel_errno, not the
real libc errno
Since the PTRACE_SET_THREAD_AREA calls and references to errno were in
kernel files, they tested kernel_errno, which is little-used, and had
some random value in it. Until -rc5-mm1, that random value happened
to be 0, and it looked like the calls were succeeding. In -rc5-mm1,
kernel_errno contained 2, so I started seeing all these nasty error
messages.
I fixed it so that the constant failures were recognized as such, then
tracked them down. It turns out that TLS entries weren't being copied
into the child process during a fork, and bogus, empty entries were
given to the child instead. Fixing this allowed me to remove a patch
which I made without understanding it, but which made NPTL and the TLS
stuff work. With that patch gone, I think TLS support in UML is
understood, and we can start cleaning it up in preparation for mainline.
15 Nov 2005
Hotplug memory is now a semi-reality. What made it so was a patch
from Badari Pulavarty which allows the punching of holes in mmapped
tmpfs files. Even as limited as this, it is exactly what I need for
hotplug memory. The way this currently works is that you use
uml_mconsole to add or remove memory in the same way that you'd plug
or unplug a device. The driver inside UML will try to allocate the
requisite memory. If it fails to allocate it all, then you won't have
pulled out the full amount. You can try again a bit later after the
kernel has had a chance to free up some memory.
You can plug in memory in the same way. The current restriction is
that you can only plug in memory that had been previously unplugged.
You can make this somewhat less onerous by giving the UML a generous
amount of memory at boot time and immediately unplugging a lot of it.
This much has always been possible. What hasn't been possible up to
now is actually freeing the memory to the host. This is what Badari's
patch does. Once it's freed on the host, it can be plugged into a
different UML or just left on the host.
The next step is a memory management daemon on the host which watches
the memory pressure on the UMLs and the host, shuffling memory around
as needed. One thing that's fairly important is to keep the host from
swapping. This makes UML performance much more predictable, as it
won't need to be swapped in to be woken up. It also avoids some
pathological swap conditions where the host and a UML swap the same
page to their respective swap devices.
The Book is nearing completion. I've sent two revisions to the
publisher in the last couple of weeks. The final manuscript is due at
the end of the month.
28 Oct 2005
So, it's been a long time since the last entry. Some of last was
laziness, some was enforced by a stolen laptop. A lot a happened in
UML-land over the last six months, and I'll mention the highlights.
I'm about done writing a book about UML. This has been taking a great
deal of my time, and it is just about done. My deadline for the final
manuscript is Nov 27. The first draft is done, and I'm currently
going through it, and the reviewers' comments, to polish it up.
It is a an how-to-use-UML book, from getting started for the first
time to setting up and running large UML servers. There's also some
history and my prognostications about the future. The publisher is
Prentiss-Hall, and it's due out in the spring.
In actual UML development, 2.6.13 and 2.6.14 (as of yesterday) are
out. A great number of UML patches are now in mainline, making it
much more robust than it had been. skas0 is now in there, along with
a bunch of associated performance improvements. More are coming in
2.6.15, notably Bodo's ldt patches.
The host AIO support went in during 2.6.14, but I yanked the ubd
driver's use of it at the last minute. There were problems in the
driver that were exposed by its use of AIO. I fixed them, but didn't
feel comfortable with them going in so late in the 2.6.14 cycle. So,
they will be introduced early in the 2.6.15 cycle in order to give
them some testing.
x86_64 has been working well for a while and now seems reasonably
mature. We are still smoking out bugs in skas0 once in a while. The
most recent one is the skas0 assembly stubs being assembled in an
unexpected way. It turns out the way I had written them (mostly with
each register assignment in a separate asm statement) doesn't
guarantee that the registers will stay that way up to the actual
system call. It also turns out that there are are asm idioms for
doing this right.
As for the lost laptop, this happened during my trip to the FSM (Free
Software Meeting) in Dijon, France. On the train from the airport to
Paris, someone sat down opposite us, and distracted us with some coins
he dumped on the floor. When we were looking under the seats, he
lifted my laptop from the overhead rack and left. I didn't discover
this until we got into Paris. The UML web site hadn't been checked
into CVS for a while (or backed up), so the only up-to-date copy of
the XML I had disappeared with the laptop. So, I grabbed a copy of
the site and gradually reproduced the XML from that HTML. This was
done a week or so ago, and I was able to start updating my patches
page.
6 May 2005
I went back to UML/x86_64 and got it working. The problem that I was
stuck on was that some processes would segfault in a way that I
couldn't diagnose. It turns out there were a few bugs in the x86_64
signal delivery code. They all were involved in extending the stack
downward when constructing a signal frame. One of them just tried to
extend the stack too far. Another, which was a generic bug, failed to
exempt signal frame generation from a stack extension check. And the
last totally mishandled a failure to do a virtual to physical
translation, resulting in signal frame data being written to physical
address zero.
With these fixed, UML/x86_64 seems healthy. I finished building a
64-bit LFS
filesystem with it. Aside from some difficulties with the packages
themselves, that went fine.
Since my 2.6 test box is my x86_64 box, and since AIO appeared in 2.6,
testing the UML AIO support has been held up on getting UML/x86_64
working. Since that is done, I went back to AIO and made it work.
Considering that the host AIO support had never been tested before, it
was surprisingly easy. A couple of bug fixes later, and it was
working. I instrumented it to see that multiple requests were being
sent to the host, and they were. I saw up to 16 outstanding requests
during a boot.
I have a ton of stuff to merge into mainline, so I started working on
that. Al Viro helped out by breaking his big cross-build patch into
manageable chunks. I sent those along, as well as about 12 others
that I had pending. I messed up a couple of them. I just forgot one
of Al's patches, on which future ones depended, so they didn't merge
well for Andrew. Another didn't go at all because I diffed it against
a built tree, and the result didn't patch into a clean tree. In the
end, those all ended up in 2.6.12-rc3-mm3, and Andrew queued them to
Linus for -rc4.
Today, I sent out 12 more patches. This time, I tested them in a
clean tree, fresh from being untarred from kernel.org tarballs and
patches. With those in, my patchset should be down to around 60.
There are more easy, independent fixes in there, so I'm going to start
doing them.
22 Apr 2005
I got my virtualized scheduler working somewhat, and announced to
LKML. Dead silence from LKML, and a bit of reaction from my group at
Intel, where I forwarded the announcement. The LKML announcement can
be seen
here.
The jist of it is this:
The virtual schedulers form sched groups, each of which is a CPU
container that competes as a single process on its host scheduler.
The processes within the guest compete against each other for whatever
CPU time the container process gets from the host scheduler.
The sched groups are visible in /proc as /proc/sched-groups/pid,
where pid is the process id of the process that made the
sched-group. These directories now contain the former
/proc/pid directories, and symlinks have been left behind for
compatibility. Initially, all processes are in sched group 0,
proc/sched-groups/0, which is the host scheduler.
The available schedulers are visible in /proc/schedulers. A process
becomes a guest scheduler by opening one of those files.
You move processes from one scheduler to another just by moving the
pid directory from one /proc/sched-groups directory to another.
The example in the announcement is three CPU hogs, one on the host
scheduler and two in a guest. They should get a 50-25-25 CPU split
because the two in the guest are competing for the 50% of the CPU that
the container process gets from the host scheduler. This is how it
actually looks:
Currently, it's for UML only - there are a couple of minor things
which I suspect will cause it not to build on x86 or anywhere else.
In other news, I brought UML up to 2.6.12-rc3. This required some
nasty hackery to get around an interaction between skas0, stack
randomization, and a consistency check in exit_mmap. Basically, the
stub data page added by skas0 was causing the check to possibly fail,
depending on where the process stack ended up in memory. Bodo and I
worked out a nasty solution this afternoon on #uml. I've been working
on getting skas0 in shape for mainline, and I pretty much had it, but
this is going to require some more work to do cleanly.
Bodo's S/390 port seems to be coming along nicely. He reports UML
working in TT, SKAS0, and SKAS3 modes. I've been merging his
non-S/390-specific patches, and waiting for him to bless the S/390
bits so I can merge those, too.
11 Mar 2005
Along with normal UML stuff, I've spent the last couple of weeks
virtualizing the the Linux scheduler. What this means is that you
have a guest scheduler which looks like the host scheduler as a single
process. This is done currently by having a process open a magic
/proc file:
cat /proc/schedulers/guest-26
This causes the cat process to turn into a new instance of the
scheduler running on top of the original. This scheduler gets
whatever cycles it can from the host scheduler and uses them to run
its own processes. This means that the group of processes within the
guest scheduler compete for cycles from the host scheduler as a single
process, making this a CPU compartment.
I gave /proc a bit of an overhaul, with all the processes on the
system starting in CPU compartment 0, which is represented by
/proc/sched-groups/0. So, all the former /proc/pid directories
start off in /proc/sched-groups/0, with symbolic links pointing there
from /proc, so that ps and related utilities continue to work.
When a process opens /proc/schedulers/guest-26 and becomes a guest
scheduler, a new entry in /proc/sched-groups is created:
usermode:~# cat /proc/schedulers/guest-26 &
Created sched_group 177 ('guest-26')
[1] 177
usermode:~# ls -l /proc/sched-groups/
total 0
dr-xr-xr-x 2 root root 0 Mar 11 17:20 0
dr-xr-xr-x 2 root root 0 Mar 11 17:20 177
This is initially empty, except for itself, because there are no
processes in this scheduling group. We fix this by moving one there -
in this case, the shell that we are typing at:
usermode:~# mv /proc/sched-groups/0/158 /proc/sched-groups/177/
usermode:~# ls -l /proc/sched-groups/177/
total 0
dr-xr-xr-x 3 root root 0 Mar 11 17:20 158
dr-xr-xr-x 3 root root 0 Mar 11 17:23 177
dr-xr-xr-x 3 root root 0 Mar 11 17:23 185
So, you can see that pid 158 is now in this group, along with 185,
which is the ls.
The next thing to get working is to have a couple of infinite loops in
a sched-group and one outside, and to see that the two inside each get
25% of the CPU and the other gets 50%. This doesn't work right now
because I don't have the timer interacting with the guest scheduler
correctly.
What does this have to do with UML? UML is going to provide the
structure for virtualizing the scheduler and other kernel subsystems.
You can make UML (or any part of it, such as the scheduler) run inside
the kernel as a guest by treating this as a new "OS". You would port
it to internal kernel interfaces rather than the libc system call
interface. This makes the "U" part of "UML" something of a misnomer,
but that's OK.
In the longer run, I'd like to be able to run a userspace guest
scheduler which would have the same properties of the current
in-kernel guest. This would use its cycles to run the (otherwise
unjailed) processes under its control. A tt-mode-like implementation
would have normal processes being run one at a time, while a
skas-mode-like implementation would have a single process constructing
the comfined processes from pieces provided by the host. So, for each
confined process, this scheduler would have an address space, a set of
register values, a set of open files, etc, and activate them all at
the same time when the corresponding process is supposed to run.
The confined processes would no longer exist in the host kernel,
except as disassociated parts. Thus, moving a process from the host
scheduler to a guest scheduler is really a process migration. I still
want the guest scheduler and its processes to be visible on the host
as they are now, so this means that the host will have to have "stub"
processes which are representatives of processes which are owned by
something else. Operations on these stubs will be passed along to the
scheduler that owns them.
This is starting to resemble a cluster, with processes migrating
from node to node, but visible across the cluster. I think that this
process of virtualizing kernel subsystems one at a time can lead to
something resembling a cluster.
16 Feb 2005
The x86_64 problem was that I was trying to use 64-bit constants and
not noticing the assembler telling me they were being truncated. I
now compute them (by or-ing two 32-bit constants together). Now, I
have to start hitting it with some I/O to see how well the AIO support
works.
15 Feb 2005
I finished the first pass of the ubd AIO support. The driver now
issues as many requests as it can, leaves them for the AIO layer to
deal with, and tells the block layer when a request is finished. It's
only tested on 2.4, which does one request at a time to the host, even
though a bunch have been issued by the driver. This is still a
reasonable test - the one thing that's different from the 2.6 AIO is
that the requests are guaranteed to finish in the order they are
issued. I think there are no issue-order dependencies, but I need to
test this on 2.6 to be sure.
However, my 2.6 box is x86_64, on which UML is currently broken. A
possible culprit is Bodo's skas0-clone patch, which makes sure that
child processes get a copy of the parent's segment registers by having
the stub in the parent actually create the child. Bodo gave me a mass
of assembly, which I gradually translated into C (with a few assembly
helpers) over the course of a couple days.
Another possible culprit is the VM op batching which I added recently,
which has the process stub perform a number of VM changes at once
instead of switching back and forth to UML for each one. I discovered
two bugs in the x86_64 support for that (one of which was also present
in the x86_64 skas0-clone support). These haven't changed the symptom
greatly, which is that the whole thing hangs on the exit of the first
non-init process. I'm seeing the two processes sharing a stack, which
is very bad, and I haven't yet tracked the cause down.
4 Feb 2005
In preparation for 2.6.11 having a good, buildable-out-of-the-box UML,
BlaisorBlade and I have been feeding select patches from my queue to
Andrew. In addition, I sent a patch to Linus (directly, not through
Andrew, which he took (!)) which fixed some mainline build problems
that didn't exist in -mm. In the meantime, BlaisorBlade pulled some
stability patches from my patchset and sent them to Andrew, who pretty
promptly sent them on to Linus.
In the meantime, the sense of urgency we feel about this has been
lessened somewhat by UML in a different way - its ability to expose
ptrace bugs. Bodo was seeing segment register corruption, and tracked
it down to a race in the host kernel where there was window during
which a parent could ptrace some values into the process, and the
process would just overwrite them due to being in the middle of a
context switch. He explained this race to me in #uml, and I got it,
somewhat dimly, after a while. He then brought it up on LKML, got
Andrew and Nick Piggin interested in it, with Andrew saying that this
was a bug to hold up 2.6.11 for.
I've started going through my patch backlog and integrating stuff from
it. The most interesting piece I did was Gerd Knorr's X11 framebuffer
driver. This gives UML a real framebuffer to use as a console, which
is pretty slick. It has to be configured somewhat carefully - until I
disabled some things and enabled others, UML first wouldn't build, and
then it built and ran, but just popped up an empty black window for a
console. This patch will fix defconfig so that the configuration is
right, and it will build out of the box.
There are some oddities with it. If you have two consoles with gettys
on them, they will both appear in the framebuffer window. This seems
to me like we need to enable virtual console switching, although it
may also be a symptom of my /dev/tty0 botch. I haven't looked at it
closely enough to tell yet.
1 Feb 2005
I spend a good part of the last week playing with AIO support in the
ubd driver. I'm doing this very incrementally in order to be able to
track down breakage easily when it happens. The first step, which
went well, was to remove the ubd I/O thread and use the existing UML
aio support, which will run an I/O thread of its own on 2.4 hosts.
Next, I tried dequeuing a request and handling it totally by myself
instead of using elv_next_request to give me a piece of it at a time.
This went less well, due to my not understanding the 2.6 block layer
changes. Once I figured out the relationship between the struct
request sector, the bio sector, and the SG offset, things went rather
better.
Now, I'm pulling a full request off the queue, turning it into a set
of scatter-gather structs, and issuing them to the host. Right now,
it's one at a time like before, but the infrastructure is closer to
being able to issue them all at once, and letting the lower layers
issue them to the host in whatever way they can.
I fell a little behind Andrew over the last couple of weeks, so I
caught up today. Merging in rc2-mm2 was fairly easy, and I'll be
pushing the current incremental patches out today.
21 Jan 2005
Bodo's segfault stub cleanups accidentally broke x86_64 by
reintroducing the original bug that had me stuck for a while. Those
changes brought back the property that UML re-enters userspace after a
page fault from inside a system call. To recap, this is bad because
system calls return using SYSRET, which returns to the address stored
in %rcx. This means that the system call wrappers can guarantee that
they don't care about %rcx, but a fault that can happen at any time
where %rcx might contain comething important can't return to userspace
using SYSRET.
I complained at him about this, and he came up with the neat
suggestion of signalling the end of the address space fixups with an
int3, which generates a SIGTRAP, but does it with a processor
interrupt rather than a system call. This returns to userspace using
IRET, not SYSRET, which solves the problem nicely.
I had my patchset down to 10 patches at one point after sending
another batch to Andrew. It's now more than that as I merge more
stuff from BlaisorBlade and keep working off my backlog. Most of this
stuff is cleanups. I split the AIO support out of externfs and
started to add AIO support to the ubd driver. Also, I'm getting UML
help from other people at Intel, and those patches have started going
into my tree. We're separating out the userspace code from the kernel
code in preparation for a ring 0 port of UML.
14 Jan 2005
I sent 28 patches to Andrew the other day. I had 50 patches in my
patchset and it was getting a bit much. This included most of the
x86_64 stuff. Amusingly, Andrew's -mm3 announcement got eaten by
vger's spam blocker, and it was my fault:
The 2.6.10-mm3 announcement was munched by the vger filters, sorry. One of
the uml patches had an inopportune substring in its name (oh pee tee hyphen
oh you tee). Nice trick if you meant it ;)
I'm not actually sure what he's talking about. I grepped my patches
for the suspect string and it wasn't there. So, maybe it was one of
his names for my patches that did the trick.
The largest things I'm still holding onto are skas0, Bodo's faultinfo
rework, and the tlb flushing improvements.
skas0 is coming along, but it still needs work. Bodo made a good
suggestion about how to pass page fault information back from the
segfault handler. This simplified the code and made it more
efficient. The original implementation was
the stub handles the segfault, and copies the fault information from
its sigcontext to registers
it stops itself
the kernel reads the values from those registers with PTRACE_GETREGS
the kernel continues the process by putting the original registers
back and PTRACE_SYSCALLing it.
This didn't work on x86_64 because of the way SYSRET works. If you
try to continue from a signal from the inside of a system call, you
corrupt %rcx. So, on x86_64, I had some special code which continued
the stub by having it return fairly normally from the signal with
sigreturn.
Bodo made the following suggestion
instead of putting the fault information in registers, just make the
stub's stack available to the kernel and have the stub write the
address of the sigcontext in a known location
have the stub hit itself with a signal that's blocked in the handler,
but unblocked outside, causing it to stop with a signal right after
the sigreturn
the kernel reads the page fault information from the sigcontext,
handles the page fault, and continues the process
This unifies the x86 and x86_64 implementations, removes some
architecture-specific definitions that used to be needed for pulling
the page fault information from the register set, and makes the x86_64
implementation cleaner and faster.
Bodo also noticed that skas0 doesn't copy ldt entries properly from
parent to child across a fork. This is not hard to fix, but does
point out that skas0 is not ready for prime time yet. I'm also
concerned about page fault speed. skas0 is noticably faster than tt
mode for everything else, but it is a couple of context switches
slower when mapping in a single page during a page fault. He made a
suggestion for this, which should speed it up a little, but still
won't touch those extra context switches. Rather than having the stub
run the system call and signal itself, it's a little quicker to just
single-step the system call. This saves the system call exit from the
single-stepped mmap and the entry for the kill(). Not a lot, but I'll
take anything I can get.
I also have patches for improving tlb flushing performance by merging
adjacent operations wherever possible. The next step, which I had
implemented, and then somehow lost, is to have skas0 batch them up so
the stub does a lot of them without switching back and forth to the
kernel on each one.
Also in my tree now are a patchbombing from Blaisorblade, so I am back
up to 40 patches. He's doing a lot of good cleaning work, and these
patches make large improvements in some areas of the code.
I need to get 2.4 back on track, so I've been figuring out what
patches need to be merged from 2.6. Sadly, I'm not organized enough
to have held onto copies of the patches I have sent to Andrew. So, I
pulled all his broken-out patchsets, and grepped out the UML-related
ones. There are 166 of them, not all of which are backportable to
2.4. However, probably most of them are, so that's what you'll be
seeing in the 2.4 part of my
patches page soon.
7 Jan 2005
I finally got the x86_64 port merged into my tree. See the
incremental patches
page for the patches. I will be sending them to Andrew when I
think they're stable, which right now, they're not. I have a busybox
filesystem which boots nicely, but when I try anything fancy, I get
crashes.
To test this, and to get a 64-bit filesystem, I've been doing the
Linux From
Scratch thing. This is a hugely useful project. I started off
by naively building gcc, then realizing I was going to need libc, and
trying to build that, and failing. Starting a toolchain and libc from
scratch is subtle, and you'll waste a lot of time if you want to
figure it out on your own. So, I now have a temporary set of tools in
my filesystem, and I'm going to continue building it from inside UML
to exercise it.
A while back, BlaisorBlade started asking questions about why
something very much like skas mode can't be implemented in stock,
unpatched hosts. It turns out that the answer is that it can. The
result is skas0 mode, which offers the security of skas mode, and a
good part of the performance, without needing to patch the host. The
two things that skas mode gives you can be implemented on stock hosts
/proc/mm - This lets you create new address spaces without needing a
new process for each one. This can't be done without patching the
host, but the benefits of this are somewhat lower memory consumption
in the host kernel. It's easy enough to get new address spaces by
creating new processes.
It also lets you modify those address spaces, i.e. mapping and
unmapping pages, and changing page protections. This can be done by
inserting code into the process, and making the process call it
whenever you need address space changes. I've done this by taking
away the top two pages of the process address space, and using one for
code that the process will run, and the other for handling SIGSEGV.
More about that later.
The code page has two pieces of UML code mapped into it. The first just
executes a system call. UML sets up the process registers for the system
call and sets the IP to the start of this bit of code, and continues
the process. The child just executes the system call and signals
itself so UML knows that it is done. This is used for mmap, munmap,
and mprotect.
PTRACE_FAULTINFO - This allows UML to get page fault information from
the child when it gets a SIGSEGV. This is done in skas0 with the
second piece of code inserted into the child. This is a segfault
handler, which reads the page fault information out of its sigcontext,
puts them in registers, and signals itself. UML then reads the
registers, and gets the page fault information from there. The data
page that is mapped into the child is just the stack for the SIGSEGV
handler to run on.
I'm trying to convince myself that this obsoletes tt mode so
completely that it can just be thrown out. From a security point of
view, it's a no-brainer. From a performance point of view, skas0 is
generally a win, but there are specific spots where it performs
worse. In particular, page fault handling is slower. tt mode can get
the page fault information directly from its stack, and fix its
address space just by calling mmap/munmap/mprotect directly. skas0
has to make two context switches in order to get the page fault
information, and at least two more to fix it. This aside, it it
noticeably faster on my favorite benchmark, the kernel build. UML can
do a build in just under three minutes on my laptop.
I've been doing some other performance improvements in order to get
skas0 performance above that of tt mode wherever possible. I've
removed some unneeded code from the system call path. I've also made
tlb flushing more efficient by minimizing the number of system calls
needed. This is done by merging adjacent operations wherever
possible. For example, a large munmap, which used to be done
page-by-page, is now coalesced and done with a single munmap.
Similarly, mmaps which are contiguous in memory and the backing file
and have the same permissions are done in one operation, rather than
one operation per page.
Having started using evolution to keep track of deadlines and to-do
lists, I've started thinking again about embedded UML as a way of
adding things that I would like, but which are specialized enough that
they probably never will be.
To recap, embedded UML is the idea of making UML into a library which
can be linked into other apps. Having done this, you would then
implement a little filesystem which would be mounted inside the UML
and give you access to internal application state. This would be
exactly like /proc, which provides access to internal kernel state.
Many entries in /proc allow reading and writing kernel variables, but
many have more complicated semantics. This "appfs" would be exactly
the same thing, except it would be specific for the application that
you're embedding UML into.
Getting back to evolution, a very simple thing I would like is some
statistics on the average age of my todo items. Are they getting
older or younger? And maybe a pretty graph showing that. I would
like to do this by making an "evofs", mounting it (on /evolution in
the embedded UML, say), looking at the items in /evolution/tasks,
pulling out their ages and doing the calculation. Lets say that each
task is a directory, which attributes for each task in its own file
within that directory. So, I could write a script that looks at
/evolution/tasks/*/start-date, calculate the age of each one, and
average them.
I realize that there's a config file with a reasonable format under
~/.evolution which can be parsed to provide the same information, but
there are other things I want which can't be had by parsing config
files.
For example, there are scheduled things which have to happen on the
same day each month, but which I'd like evolution to schedule on a
weekday if that happens to be Saturday or Sunday on a given month. I
don't necessarily fire evolution up during the weekend, so the alert
will show up on Monday in those cases, which would be non-optimal. If
it alerts my on Friday, I make a note on a Post-It and have some
chance of remembering it during the weekend.
Needless to say, there is no "put this on the 15th of each month,
except if that's a weekend, in which case put it on the previous
Friday" button. So, what I want to do is have a little daemon in my
UML inside evolution which watches for new appointments by watching
/evolution/appointments via dnotify or inotify. When one shows up on
a weekend, it would just just change the date field to the previous
Friday. There are obvious problems with appointments which are
scheduled forever into the future. One of these can't make an
infinite number of /evolution/appointment directories for this daemon
to look at. Evolution could just make directory entries for
appointments that are visible in the interface. So, if you go
scrolling through the rest of the year, everything you see will be on
a weekday.
Another example - I'm starting to write a book about UML, and just
finished the schedule for it, i.e. when each chapter will be ready. I
had a similar problem with this. Each deadline is some number of days
later than the previous one, and some of those will end up on a
weekend. I wanted each deadline to be on a weekday, except the
nearest one this time, so a Sunday deadline gets moved to Monday. A
similar daemon could be used to do this - the logic would be quite
similar.
However, we can't have two daemons battling over the same
appointments, so we need some way to specify which private rule will
be applied to an appointment. The best thing would be some button in
the appointment dialog that says "If this appointment falls on a
weekend, it should be moved to the previous Friday" or "If this
appointment falls on a weekend, it should be moved to the nearest
weekday". Evolution can export its internal state to an internal UML
via a filesystem, so it can export its UI in the same way. So,
somewhere in /evolution/ui, there'd be a directory for the appointment
dialog box. Fiddling that in the appropriate way, i.e. creating
directories or files for the new labels and checkboxes with the
information needed to position it correctly, would cause additions to
the UI when you next create an appointment.
However, if you do check those boxes, evolution won't do anything
about it because it knows nothing about those widgets except that you
told it to make them. But part of the information that you plugged
into the UI could tell evolution to include the final state of those
widgets in a particular place within the appointment's directory. So,
when it set up the checkboxes, my script could tell it to put their
state in the nearest-weekday and previous-friday files and could look
at /evolution/appointments/00123/nearest-weekday and
/evolution/appointments/00123/previous-friday to tell whether I
selected either one and move it appropriately.
Admittedly, these are minor things to use to justify a non-trivial
project such as making UML embeddable, and then modifying applications
to embed it. However, there are much more substantial uses for this
that I can imagine. For example, the embedded UML can export
application filesystems to the outside world via NFS or some other
network filesystem. In a group environment, this could mean exporting
your task list to your boss's evolution, where his UML will make a big
list by glomming together all the underlings' lists. The individual
could be mounted under /tasks/alice, /tasks/bob, etc, and then
symlinks made from /evolution/tasks to tasks within those
mounts. The "taskfs" running that filesystem would see the symlinks
being created, read the contents, and create the datastructures inside
the application to make all those tasks appear within the boss's UI.
As described above, the UI could also contain additions, such as the
owner of a task, that the original UI doesn't have. Then, there could
be an interface for changing it. Again, this would be completely
implemented by a script within the embedded UML. Changing the owner
of a task would involved moving it from one person's list to
another's. From the point of view of the script, this is moving the
task's directory from /tasks/bob to /tasks/alice. Then, Bob's and
Alice's UMLs will find out, via NFS, about the moved file, make the
changes locally, and by doing so, inform their respective evolutions
so that one will show one more task in its UI and the other will show
one fewer.
I don't know if evolution has this sort of thing already, or if it's
planned. If not, this shows how embedding a UML into an
application can make it much easier to extend, possibly in fundamental
ways, such as making an isolated application group-aware. You get a
standardized interface, the Unix file interface, and all the Linux
tools for using that interface in all applications that embed UMLs.
You don't need to worry about the application's source, or building
it, or the language that you need to use. The one place where you're
dependent on the application is what information it exports via a
filesystem and what you can do through it.
18 Oct 2004
UML updates have been going nicely into 2.6 via Andrew, so much so
that 2.6.9 won't be that different from my own tree. Thanks to
Blaisorblade for pushing patches to Andrew as well. He pushed the
initial set that got UML back into -mm, plus a batch just under the
wire for 2.6.9-final.
This leaves the question of what happens to the separate 2.6 UML
patches. Right now, my plan is to stop producing them, and let people
use the UML in -mm or -linus. People who want the latest
bleeding-edge stuff or a bug fix that hasn't made it to Andrew or
Linus yet can just grab it from my
patches page.
The other major UML happening is getting the x86_64 port merged into
my tree. The results of this can also be seen on my
patches page. I've
got most of it merged now - the exception is the 32-bit compatibility
code, a lot of which is gross, and it's optional, so I'll merge that
after the main body of the patch is merged and working.
So, I'm currently trying to get it to build. It looked ugly at first,
but fixing the most common compilation errors made the rest look a lot
more manageble.
23 Sep 2004
My patches are continuing to flow nicely through Andrew into 2.6. At
this point, 2.6.9-rc2-mm2 is almost the same as my 2.6 tree. There
are still some patches to go, but it's pretty close.
I'm working on getting the other major outstanding patches merged.
BlaisorBlade sent me a good number of his patches, and I've got them
mostly merged now. He did a bunch of good work on the build. make -j
now works nicely, plus UML does its final link in the same way as the
other arches. His patch did away with linux being the default target,
but I added that back. You now get a vmlinux and a linux, which are
the same thing. I sent about half of these to Andrew tonight, so they
should be appearing in the mm tree shortly.
Another of the outstanding patches is the x86_64 port. I spent
yesterday breaking it up into smaller patches. So, now I have 29
patches that I will be merging in. I have three of the smaller ones
in now. They are available from the new -mm section of the
patches page.
I also revived the skas4 patch. I'm having confusing problems with
UML host processes getting signals that I think they shouldn't.
Debugging on this is continuing.
After all these are done, or maybe in parallel, I'll start looking at
Gerd Knorr's patchset, plus smaller things like meta_tdb. This will
knock the patch backlog down nicely, and then it will be possible to
start looking at the small things that people have sent in, and which
have been languishing.
11 Sep 2004
This month's big news is that Andrew has started feeding the UML
patches he has been keeping in his tree on to Linus. BlaisorBlade
sent the big patch plus some patches of his own to Andrew, and I sent
in some more which made it build and work. At that point, which was
sooner than I expected, Andrew sent the whole thing to Linus, who,
miraculously, took it.
There were some glitches. Among them were my patch changing the
initial value of jiffies from -5 minutes to 0, which I put in because
it was getting a strange value in UML, another which added a UML
patch number to EXTRAVERSION, and, of course, ghash.h. The first two
were easily fixed, and I ripped ghash out of UML the day before
yesterday.
I've also started feeding him the patches that make up the difference
between what he has and what I have here. So, UML should be
reasonably up to date in -mm, and also 2.6, pretty soon.
BlaisorBlade made a suggestion which I think is reasonable, which is
that -mm and 2.6 should be the UML stable tree, and my tree should be
the development tree. Thus, patches would be forwarded from my tree
to Andrew when we think they're stable, and my tree would consist of
so-far-inadequately-tested patches. I'm also considering stopping
releasing 2.6 patches. You'll have the 2.6 tree with a good UML in
it, and my ongoing development will be available as patches from my
incremental patches
page.
I've also got externfs/humfs ported to 2.6 and working reasonably
well, except for some internal glitches, which will be fixed. With
some more work to enable mmap in humfs, this will let us start
eliminating the excess memory consumption caused by double-caching on
the host and UML.
19 Aug 2004
I've been keeping up with 2.6. I put out a second 2.6.7 patch which
syncs up my 2.4 and 2.6 trees. Then, I put out a 2.6.8.1 patch which
just updates to 2.6.8.1. I'm also tracking -mm in hopes of cleaning
up the UML in Andrew's tree enough that he can send it to Linus, and I
can finally get something current into stock 2.6.
I spent the last couple of days writing my views of where UML is
headed in the future. It is somewhat grandiosely called the Road Map. It doesn't
contain anything that I haven't talked about before, but that is all
buried in the papers and slides on the site, and I think that very few
people read them. This puts it all in a much more conspicuous spot on
the site.
Most people, understandably, consider UML to be just a virtual machine
technology. I'm trying to make clear that virtualization in general,
and UML in particular, are potentially much more than that. I talk
about porting UML into the kernel, and the possibilities that
creates. Also, UML can be linked into other applications and used as
a captive virtual machine and I describe what I see as some of the
possibilities there.
12 Aug 2004
The Kernel Summit and Ottawa Linux Symposium have come and gone.
Virtualization was a pretty consistent theme in both, and UML came up
often. At KS, Chris Wright gave a talk on virtualization on Linux in
which he described the various technologies that are currently
available. Then he talked about what Linux could do to better support
virtualized guests. Since I had given him most of the material for
this part, he more or less turned it over to me, and I went through my
laundry list of things that the host could do to make life easier for
UML. None of what I said seemed particularly controversial, which is
nice. Wim Coekaerts of Oracle was particularly interested in UML's
need for AIO, since Oracle uses it, and Wim perceives a lack of
interest in it on the part of Linus.
At OLS, there were a number of virtualization talks. An IBM Power
guy, Dave Boutcher, gave a talk on hot-plug CPUs and memory. The CPU
stuff is going to be used by UML in exactly the same way that the
other arches will. However, his memory hot-plug plans were aimed at
being able to remove a particular piece of physical memory from the
machine. This requires moving all kernel data structures from that
memory (or not putting them there in the first place). UML's memory
hotplug plans are much simpler. It is sufficient to be able to grab a
number of pages and free them to the host, and it doesn't matter where
they come from, or that they are contiguous.
There was a talk by the Xen people, which I missed because I went to a
different talk. Chris gave his Linux virtualization talk again, which
I punted on this time because I had heard it already. I don't know
what he did with the section that I pretty much did during KS.
On returning from Ottawa, it was time to get back to UML, which I can
now do pretty much full-time, thanks to Intel. Since then, I've got
UML pretty much up to date with all the trees I keep up with. I fixed
a bunch of problems in the latest 2.4.26 release, released a 2.6.7
patch, and, last night, got 2.6.8-rc4-mm1 running and sent the
requisite patches to Andrew.
I've started using quilt to manage patches, and started posting my
current patchsets to the
UML site. This has gotten good reviews from my users, and has helped
me by giving me some assurance that I'm not going majorly break
anything when those patches are released for real.
I currently have my 2.6.7 tree synced up with 2.4.26, except that
humfs and hostfs aren't all that stable yet after being ported over.
The mm tree has a pile of bb patches in it which I need to sync up
with, and merge any applicable ones into 2.4.
15 Jul 2004
Recent UML work has focussed on the host-based filesystems, mainly
humfs. The filesystem metadata has been reworked to fix some bugs in
the first version, namely to add modes to the metadata, handle hard
links correctly, and to correctly handle the case of where to put the
metadata of the parent directory containing a "metadata"
subdirectory. I had a plan for files named "metadata", but forgot to
check that it handled directories as well. The fix is to create two
metadata directories, one for files and one for directories. In the
directory metadata, there can be only one plain file in a given
directory. This will normally be called "metadata", but if there is a
directory by that name, it will be called something else. Whatever it
is called, it can be found by looking for a non-directory.
This required bumping the metadata version and updating humfsify.
Since I don't want to increase version numbers very often, I also
fixed the mode problem, which is also a version-affecting bug, at the
same time.
Humfs seems healthy at this point. I can boot from it and do a kernel
build on it. Hostfs was also affected by the externfs restructuring,
and got little attention, so it was broken. So, I've been working on
it. It is better, but there are still some bugs. It doesn't survive
a kernel build yet, but it's doing a lot better than it was yesterday.
I've moved UML development onto my new laptop, which means I'm running
UML in tt mode until a put a skas kernel on it. This exposed a
problem with the file descriptor management that I wouldn't have seen
otherwise. In tt mode, a pipe is used as part of context switching.
The outgoing process writes a byte into the pipe of the incoming
process and then reads its own pipe. The write will bump the incoming
process out of its own read, waking it up, and the outgoing process
will sleep in its read until something writes to its pipe.
The problem was that the pipe file descriptors weren't under the
control of the filehandle code, so it couldn't free up file
descriptors if the pipe call failed due to no descriptors being
available. This required a bit of surgery on some tt-specific code,
which seems to be working well.
30 Jun 2004
I've had precious little free time in the last few weeks, but what
there was I spent banging on humfs. It now stores file modes in the
metadata rather than leaving them on the original files. humfsify now
has a -r option to merge the matadata back into the data. This is how
you upgrade from one metadata version to another (or change metadata
types) - revert the metadata and build the new metadata from scratch.
The major development, and what sucked up a fair bit of that free
time, was a nice offer from Intel, which I accepted. I get to spend
most of my time working on UML, so this means that it is no longer a
free-time project. We should see UML development speed up noticably
as a result of this.
However, that probably won't happen until the end of July. I'm taking
off next week and touristing around Iceland. Then, two weeks later,
its KS and OLS. There is that week in between, but I'm expecting that
to be largely spent catching up on what I missed the previous week.
So, that means that it'll be the last week of July before I really get
cranking on UML again.
I am trying to get a release out before I leave. The last patch is
noticably less stable than the previous ones, so I'd like to get a
bunch of bug fixes in.
13 May 2004
After much banging on hostfs and humfs, I got them working well enough
to release them. It turns out that there were a bunch of bugs in
hostfs conspiring to make it completely synchronous. This is
undesirable for something like humfs, so I made it behave like the
disk-based filesystems. That exposed a pile of bugs, which I fixed,
now humfs is noticably faster. I also added a patch from Piotr Neuman
which makes it possible to plug new metadata types into humfs. So, it
will be able to support all the other metadata formats that people
have been suggesting, like tdb and xattrs. So, that made up the bulk
of 2.4.24-3, which I released yesterday.
In somewhat older news, there is now a x86-64 port of UML, which was
sponsored by PathScale. This is on 2.6
only right now, and it's available
as a separate patch to be applied on top of the 2.6.4 UML patch. I'm
going to be merging it into my tree bit by bit. Currently, it's not
very clean in a bunch of ways, and there will need to be some work to
make it cleanly mergable. So, as this happens, the separate patch
will shrink, and when the port is fully merged, it will go away totally.
Now that the humfs work is out, I need to catch up with 2.4 and 2.6.
So, first up is 2.4.25, which turns out not to need any UML changes.
That will be released today. Then will be 2.4.26, which I think just
needs the cmpxchg patch which has been floating around for a while.
Then, it's on to 2.6, where I am currently two released behind.
19 Apr 2004
It turns out that humfs was fairly buggy. I stomped out enough of
the bugs that you can now do a kernel build on it. There are also a
couple of design problems with the current metadata layout. One was
pointed out to me by Paul
Wagland; the other I actually figured out on my own.
The first is that permissions are kept on the original file. The
problem is that if it is chmod-ed 000, then even the owner can't read
it. However, root can, so this would lead to a situation in which
root inside the UML couldn't read some files that it should be able
to. The obvious solution is to have humfsify chmod all of the files
777, and move the permissions into the metadata file.
The other bug is that the current humfs metadata maintains one file
for each name in the filesystem, so for files with multiple links,
there is an independent metadata file for each link. This is wrong,
since if you change the ownership of a file through one link, the
ownerships of the other links should change as well. humfs will
currently change the ownership of the one file, but leave the others
as they were.
My current plan on fixing this is to designate one metadata file for
each file as the primary metadata holder. If the file has more than
one link, the other metadata files will be symlinks to the primary.
This means that humfsify will need to keep track of what names link to
what files so that it knows when it needs to start making symlinks,
and where they should point to. This sounds nasty, since the easiest
implementation involves keeping track of every file it deals with, but
a simple optimization is to just keep track of the files with more
than one link. This will be a small minority of the files. Some of
them will involve links from outside the hierarchy being copied, and
won't result in any metadata symlinks. The other side of this is that
the metadata file will need a link count and a deletion flag in it.
These are needed because someone might remove the name corresponding
to the primary metadata file. That file can't actually be removed
because the symlinks count on it, and we don't want to move the data
because that would require searching for every affected symlink. So,
when the file is deleted, the metadata file will have its link count
decremented, and the deletion flag set. Then, it will appear not to
exist. It will be really deleted when all of the links to it are
gone. Deleting one of the other links will just involve decrementing
the link count in the primary, and removing the symlink.
Also in the not-too-distant future is O_DIRECT and mmap support for
humfs. Both will eliminate the double-caching problem that prompted
this filesystem in the first place. O_DIRECT is useful when the data
isn't shared with any other UMLs, which will be the case until humfs
has COW support. It reads data from the disk directly into the buffer
provided by the process without going into the kernel's page cache.
The other attraction of O_DIRECT is that it is supported by the
current 2.6 AIO. This means that UML can have many I/O requests in
flight on a 2.6 host using O_DIRECT, which should help its I/O
performance.
mmap support will do the same for shared data. In this case, the data
is in the host's page cache, and it is mapped directly into the UML
address spaces. This will need COW support in humfs before it becomes
useful. It will also need AIO support for normal, cached, I/O before
UML's use of AIO can come into play.
I'm planning on releasing a new UML patch in the near future. It will
have a fixed humfs, plus maybe a restructuring on humfs to allow
multiple forms of metadata to be supported. There will also be a
utilities release for a rewritten version of humfsify.
7 Apr 2004
It's been a while since the last entry. Since then, 2.6.4 came out,
and I released a 2.6.4 UML. I also released 2.4.24-2 today, which
contains some major changes.
I became convinced (by Al Viro) that ubd-mmap wasn't viable. It's
vulnerable to the filesystem making changes to data that it intends
never to reach the disk. With buffers mmapped from the host, those
changes automatically read the underlying device. He suggested a
filesystem instead, which gives me the control that I need in order to
do mmap correctly.
So, humfs is out. humfs stands (roughly) for "host uid mapping
filesystem". This is the main difference between it and hostfs. The
main problem with hostfs is that any files created are owned by the
uid that is running UML. This is a problem when there are multiple
users inside the UML creating files. The files want to preserve their
ownerships, but can't.
humfs separates the ownerships from the actual file. A humfs mount
has a root with two subdirectories, "data" and "metadata". "data"
contains the actual files. "metadata" mirrors "data", except that the
files under it contain just ownership information as their contents.
humfs file access cause the file contents to be retrieved from under
the "data" directory, and ownerships to be retrieved from under
"metadata".
This has a number of beneficial side-effects:
As already mentioned, mmap can be done correctly. This will allow
files to be mmapped from the host rather than being copied, allowing
UMLs to share the host's page cache. In the tests I've done with
ubd-mmap, this dramatically reduces the host memory usage of a UML.
The UML's files are visible on the host. This makes management
easier, since passwords can be more easily reset and the like. There
are some privacy concerns here in hosting environments. Some people
don't like it to be too easy for the host admin to poke around their
files. So, this will have to be balanced against the management
conveniences.
Space is more flexibly assigned. There is a "superblock" file at the
root which says how much space this mount is allowed to have. It's
read at mount time currently. I think that with dnotify, it will be
possible for humfs to notice changes to it and react immediately. So,
on-the-fly disk allocation changes will be possible.
File-level COWing will be possible. This will allow UMLs to share
filesystems, with modified files being copied and made private. A
special case of this is booting from the host's root filesystem, with
a tiny COW hierarchy containing the files which need to be different.
Block-level COWing within a file will also be possible. Some people
need block devices underneath their UML filesystems, and here, they
will want the usual block-level COWing. I think it will be possible
to do this while retaining the memory usage advantages of mmap by
having the disk image in a humfs filesystem and loop-mounting that as
the root filesystem within the UML. I was concerned about the lack of
partitioning support in the loopback device, but I recently saw a
patch on LKML which fixes that.
It will even be possible for UMLs to share a writeable filesystem,
with communication allowing one UML to change a file and cause the
others to invalidate their cached copies of it. I'm planning on
figuring out how to make it possible to quickly bring up UMLs in
response to a load spike of some sort, like a slashdotting on a web
server. Coherent filesystem sharing could play a role here. With the
filesystem on a SAN of some sort, the new UMLs could serve the data
from a common source, and write things, like log files to a common
place, eliminating the need to collect them from the UMLs as they are
shut down when the spike passes.
As a side-effect of this, I put an abstract interface between
hostfs_kern and hostfs_user which allows other userspace modules to be
plugged into hostfs_kern. humfs is obviously the first new user of
this. Now that there's a pluggable interface, it's not too hard to
make other host resources look like UML filesystems. Some examples:
sqlfs - mount a SQL database as a UML filesystem. There are lots of
choices for mapping part of the database onto Linux directories and
files. Tables could be top-level directories, rows could be
subdirectories, and columns could be files within those
subdirectories. Or rows could be files. Or both, I think, depending
on whether you cat the row or cd into it. Plus, there'd be a /query
directory in there somewhere which would let you treat a query as a
directory, and the search result would appear under it. Consider
cd /sql/query/'select * from people where first-name = "Bob"'
Obviously, any other sort of database could get the same treatment.
Other people have mentioned ldap, for example. An interesting
variation on this would be to dump a Linux filesystem into a database,
and boot from it. This would give you the standard Linux file
semantics, but would also let you database-specific searches on the
filesystem. For example,
cat /sql/"select filename from * where uid = 0 and setuid = 1"
A possible use of this would be for an application which needs a
filesystem to store stuff, but whose needs are otherwise badly served
by any existing filesystems. Maybe it needs a few huge files, or a
huge number of tiny ones, or huge directories, and current filesystems
don't supply the searching that's needed. You'd put those files in a
database, mount the database inside UML as a filesystem, and provide a
query interface to directly do searches on the database.
difffs - mount a directory diff as a filesystem. The two host
directories to be diffed would be arguments to the mount command. The
files in the resulting filesystem would be those that were different,
and the contents would be the diff of the file.
googlefs - mount Google as a filesystem.
cd /google/"user-mode linux"
could produce a directory in which Google's results for the search
"user-mode linux" are somehow represented.
Some of these examples are far-fetched, but I mention them to show the
range of possibilities there are with this. I'm particularly
interested in the filesystem-in-a-database. I imagine that there are
new things that you can do when you can put an arbitrary database
underneath your filesystem, and be able to query the database in
whatever way it allows.
I also added aio support to the os interface. The existing aio was in
the ubd driver, and consisted of a separate thread which did
synchronous IO, one request at a time. This is present in the new
aio. What is new is support for the 2.6 aio interfaces. This allows
any number of IO requests to be started, and for one thread to handle
their completions. When this is debugged, this should help UML's IO
performance.
With mmap IO in sight, the next step will be to introduce the active
UML memory management that I've been working towards for a while. In
conjunction with the /dev/anon patch, mmap support will allow pages to
be freed from a UML to the host, and conversely, be plugged into a UML
to increase its available memory. With a daemon on the host
monitoring the memory usage of the UMLs and the host, it will be
possible to use memory more intensively by giving it to UMLs that need
it and taking it away from those that don't. This should increase the
density of UML servers since memory is often the bottlenck.
15 Feb 2004
As of today, I'm caught up on both the 2.4 and 2.6 fronts. I released
the 2.4.24 patch today. It was trivial, as the 2.4.24 patch was
trivial - just apply the patch and rebuild. I also had a bunch of
accumulated bug fixes, including some time problems that I fixed in
the last few days. These are the ones which people have noticed the
most. They included
process start times as shown by ps drifting away from clock time
/proc showing a modification time of 1970
at reporteedly not working
The other user-visible fixes were a couple of mconsole bugs.
I also caught up with 2.6 - more completely than I had intended. I
pulled his tree, not noticing that it contained 2.6.3-rc2 rather than
just 2.6.2. So, I ended up updating UML to that rather than trying to
back down.
I've also been busy on
usermodelinux.org
. There were a number of things from UML users that the wider
community needs to know about, from new filesystems and installation
HOWTOs, to an automated UML network setup tool, to a neat Knoppix
image which lets you download and boot UML from a web browser. I also
added some new FAQ entries and UML hosting providers. Hopefully the
FAQ entries will cut down on duplicate list traffic (and frustrated
users who just quit rather than ask a list).
16 Jan 2004
I forgot to mention that I also released my /dev/anon patch. This is
a special device that UML can use to map in its physical memory which
has the semantics needed to free memory back to the host when UML
isn't using it any more. In some quick testing, it reduced UML memory
usage on the host by about a quarter. It's used in conjunction with
ubd-mmap, but it's also usable eithout it. ubd-mmap still has some
bugs that need rooting out, including at least one file corruption
bug, so it shouldn't be used for production yet. See
this page for more details.
15 Jan 2004
I was somewhat behind in getting 2.6.0 working and released. But,
that's done now, and I also caught up with 2.6.1 reasonably quickly.
2.6.1 was causing process segfaults with the 2.6.0 patch, and the
reason was somewhat interesting. I had a couple of inline functions
which looked at the frame pointer and stack pointer. In order for
these to work correctly, they really needed to be inlined because they
needed the frame and stack pointers of the caller. Something changed
in how gcc is invoked which caused it to stop inlining these
functions. So, the fix was to turn them into macros, which gives the
compiler no choice about inlining them. This may also explain some of
the wierd behavior other people have been seeing, and which I've been
attributing to the lack of get_thread_area and set_thread_area.
Andrew included my 2.6.1 patch in 2.6.1-mm4. Hopefully it will then
make it into 2.6.2. There will still need to be a separate UML patch,
but it will be much smaller. There are some difficulties in initcall
ordering between the UML console and serial drivers, and the generic
tty driver which require some kludging. So, at least that will need
to be separate for a while until something cleaner comes up.
I also made the first utilities release since the fall. A number of
small changes and fixes had accumulated. There were two sizable
changes - bridging support in the jail.pl script and support for
dumping tty log output into a SQL database rather than to the
terminal. These both stem from my UML honeypot work at Dartmouth.
15 Dec 2003
Richard announced his implementation of swsusp for UML a couple of
days ago. Reaction is less than I expected, but still positive, as I
expected. It'll be more widely used once it is updated to the latest
UML, and doesn't require a 200M download in order.
I've been hacking away on /dev/anon recently. This is a new driver
that supports mmap/munmap, and which releases memory that is no longer
mapped. The purpose is to make ubd-mmap useful by freeing UML physical
memory which has been over-mapped with pages from a disk. With a
simple test of booting my Debian testing image to a login prompt, this
consumes about 25% less memory than using the usual /tmp file.
Accordingly, I can boot about 25% more UMLs before the host starts
swapping. When I release it for general consumption, you can all
thank memset
for the reduced memory usage (and swapping) on your hosts.
I'm also releasing 2.4.22-7. This is a big patch, with a bunch of new
stuff in it. The big items are partial support for skas SMP and
highmem, and Sapan's real-time clock patch. There are lots of smaller
changes including bug fixes and a bunch of code cleanup and
restructuring.
4 Dec 2003
I spent a week in Japan, courtesy of Richard Potter at the University
of Tokyo. He and others in Japan are doing some interesting things
with UML and virtual machines. At dinner on the first night in Japan,
I discovered that Richard had implemented swsusp for UML. This is
obviously a welcome surprise, given that swsusp is probably the most
asked-for feature for UML. The downside is that he is working with an
old UML (2.4.18). Updating it to the latest shouldn't be too hard,
though.
He also had some other interesting ideas, including the idea of a
UML-wide fork, which would be a cloning of the entire state of a UML
into a new UML, with shared resources being COW-ed as necessary. This
is exactly analogous to a process fork, except you get a whole new UML
rather than a new process. I'm not sure what I would use this for if
I had it, but I think it's worth doing, just so the rest of the world
can figure out how to use it.
I also visited the University of Tsukuba to give a talk to the group
led by Prof. Kato, and to listen to a set of presentations on their
work. My talk was a stripped down version of my UMich talk. Their
work included
a "Software Pot", which is a controlled environment for executing
arbitrary, untrusted applications with the ability to import resources
from the host
a very minimal virtual machine environment which can boot slightly
modified Linux and *BSD kernels
A Knoppix environment which loaded off the network, rather than a CD,
and booted inside a UML.
I repeated this talk the next day at the University of Tokyo (and a
number of the Tsukuba people came to hear it again), plus I gave a
presentation to the Yokahama LUG.
10 Nov 2003
After much delay and items accumulating in my todo list, I released
2.4.22-6 today. This contains a large number of fixes and cleanups,
mostly sent in by users over the last few months. One notable bug
which I think is fixed is the "Process nnnnn exited with signal 11"
that Oleg has been seeing. It was a longstanding, stupid bug, and I'm
amazed it hadn't been seen sooner.
6 Oct 2003
Last week, I visited the University of Michigan at the invitation of
Peter
Chen, who has a research group doing interesting things with
virtual machines. The one thing that I'm particularly interested in
is a tool called
ReVirt, which can log everything happening in a VM
and replay the log so that the replay causes exactly the same
instruction stream to be executed. This would make it possible for
someone with a bug that I can't reproduce to send me the ReVirt log,
and I would replay it (over and over if necessary) until I tracked
down the problem.
There was another tool whose name I forgot (written by Sam
King) whose job it is to analyze these logs. The work of this
group is focussed on security, so ReVirt is used to record attacks and
exploits, while the log analyzer pulls out the essential details of
those attacks and generates a nice picture of them. The analyzer
works by defining a small set of basic objects, such as files and
processes, and a set of actions by which they can affect each other (a
process creating a file, or a process being created by execing a file,
for example), and taking the endpoint of an attack (such as a running
backdoor process, or a modified passwd file), and backtracking through
the set of events and objects involved in creating that endpoint until
it identifies how the exploit entered the system. Then it generates a
little diagram which serves as a picture of the exploit.
The use of ReVirt in UML development is obvious. I'm wondering
whether the analyzer could be repurposed with a different set of basic
objects and actions in order to analyze kernel bugs. For example,
when chasing a deadlock, we could make locks be the basic object and
lock and unlock be the basic actions, and get a diagram showing where
the lock in question was taken and released. Hopefully, there would
be a glaring mismatch which would identify the bug at a glance.
These two tools were originally done using
FAUMachine (formerly UMLinux). At the time, there were some
reasons that FAUMachine was preferable to UML (UML's use of helper
threads complicated things) but the group is now porting them to UML.
UML is now seen as preferable because of its greater stability and user base.
FAUMachine's goals are apparently less of a good match to their
research than UML's goals. Plus, the threads issues need to be dealt
with at some point anyway.
I also spent a day at CITI, whose research
doesn't directly involve virtual machines. They are into research and
development which can benefit from using them as their development
platform. At this point, they are heavily into NFS V4 development.
They are producing the reference implementation at the behest of Sun
and
Network Appliances. They are also doing research of the
replication and load balancing aspects of the NFS V4 protocol. In
some of these projects, a virtual UML network would make an ideal
platform, since it would eliminate a lot of logistics in setting up
and running a physical test network.
18 Sep 2003
I released 2.4.22-5 today. I've been doing lots of code cleanup
lately. That was pretty much all of 2.4.22-4 since Steve Schmidtke
sent me a large cleanup patch. A lot of -5 is dealing with the
aftermath of that. There were a few bugs in Steve's original patch,
plus I merged a couple of chunks badly. In addition, there were some
more fixes from him and BlaisorBlade merged. I also fixed a tt mode
bug which caused signals to be disabled in userspace.
11 Sep 2003
HP sent me a nice IA64 machine, which showed up yesterday. Dual
processor, 10G memory, 70G disk. It's a very fast machine, at
least compared to my existing hardware, which is all ~3 years old, so
I don't have a feel for how fast PCs are currently. I spent the
afternoon turning it into my main workstation. The current holdup is
that pppd doesn't work. It gets EFAULT whenever it does the PPP
get-unit-number ioctl.
I guess I have no excuse for not at least thinking about porting UML
to IA64. I have little free time, but I'll do the port a bit at a
time in my spare moments.
8 Sep 2003
I released 2.4.22-2 today. It contains a bunch of bug fixes and a new
mconsole command. New to mconsole is the 'proc' command. This was
triggered by a patch I got from Steve Benson which implemented 'mem'
and 'load' commands, which sent back something that resembled the
contents of the UML's /proc/meminfo and /proc/loadavg. The patch
implemented them by hand, which I didn't like, plus I think it is
likely that it would have triggered requests for other specific stuff
from /proc. So, I just added a general 'proc' command which will read
from any file in the UML's /proc.
I fixed a few nasty-looking bugs such as a crash when running a UML
that had been linked against a libc with the new thread support and a
crash caused by a process unexpectedly segfaulting. It also turns out
that ltrace never worked in skas mode. I fixed all of these, plus a
couple more minor things.
3 Sep 2003
I've made a bunch of UML releases in the last few days. I added mmap
support to the ubd driver, which allows it to use mmap instead of read
and write. This trades data copying for TLB operations, gaining some
CPU and cache improvements from not having to copy all data coming in
from disk in exchange for the added expense of TLB flushing. An
upcoming benefit, and the real reason for this, is that this will
enable the UML and host to share page cache rather than having
separate copies of the data read in by the UML, reducing the host
memory usage by having the memory overlaid by mmap freed on the host.
I realized fairly recently that the host mechanisms that can be used
to implement UML physical memory won't free memory when it's
unmapped. The reason is that the memory will be considered dirty by
the host, even the the UML has no use for whatever is in it, so it
will be kept, and possibly swapped. What I need, and what isn't there
yet, is something that will just free the memory when UML unmaps it.
So, I'm going to be hacking on a little host driver specifically to
implement UML memory.
This required a rework of the UML low-level VM layer, including the
ability for physical memory to fault and for those faults to be
handled. This is something needed by a clustering scheme for UML that
I thought up a long time ago and never did anything about. That is
taking an SMP UML and spreading the virtual processors and physical
memory over multiple hosts. Any physical page would be resident on
only one host and accessible only by the processor(s) running on that
host. If another processor tried to access it, the access would
fault, and a low-level handler would figure out where that page was
located and arrange for the contents to be copied and the page to be
mapped on the new host and unmapped on the old one. This would fairly
easily implement a SSI cluster with UML. The downside is that it
would be horrendously slow because of all the faulting. The
attraction of it to me is that, even in its slow form, it would be a
neat capability and fun to play with. Further, since this starts as a
fully-functional SSI cluster, it would be fairly easy to start working
on making the inter-node communication more sane. It would be done
incrementally, with a functioning (and hopefully faster) cluster at
each stage. It wouldn't be necessary to implement (say) 90% of it
before anything works at all.
I also implemented the COW V3 format, which fixes a number of problems
that had cropped up with V2. The most painful one was the rounding
bug which has been killing UMLs for some time. David has been
maintaining a patch for it, but it apparently is not a 100% fix. The
V3 fix should be. The various sections of the COW file are now nicely
aligned. This will allow COW files to be stored on devices with
restrictive alignments, such as /dev/raw devices. ubd mmap also
requires this because it needs the data to be page aligned.
After this, I decided to catch up to Marcelo, and updated to 2.4.21
and 2.4.22. These were both easy. I also included a couple of small bug
fixes in the 2.4.22 release.
17 Aug 2003
I spent the afternoon yesterday tracking down the task_struct leak in
2.6. Oleg had tracked the leak down to where it was happening by
identifying the put_task_struct call that should have been called but
wasn't. He got that right, but his analysis of the leak was wrong.
So, I finally tracked it down to what appears to be a bug in the
scheduler combined with my low-level switch_to() being slightly
different from the i386 switch_to. context_switch calls the arch
switch_to:
switch_to(prev, next, prev);
return prev;
and this, contrary to appearances, is supposed to reassign prev, which
then gets returned to schedule(). In the i386 scheduler, I don't see
prev getting reassigned. It does branch to __switch_to, which returns
the correct value for prev. This is left in %eax, and accidentally
becomes the return value of context_switch because that's the last
thing it does, so nothing munges %eax afterwards.
So, the UML bug was that my switch_to wasn't assigning prev. Once
this was fixed, the task_struct leak disappeared.
I had a couple of brainstorms yesterday, too. I was looking over a
patch that someone sent me which made load and memory statistics
available through mconsole. I don't like it because it duplicates a
bunch of code from elsewhere in the kernel into UML. It occurred to
me that it would be easy to make all of /proc available through
mconsole in a much cleaner way.
The other thought relates to UML's low-level memory management. It's
reasonably easy to pull memory out of a UML. It can just allocate
pages, and do nothing with them. Those pages can be added back later
just by freeing them, making them available to the rest of the UML.
So, a UML's host memory usage can be controlled by having it pull
pages out of service and freeing them back to the host. This would
make a good way of taking pages from an idle UML and giving them to a
UML that's busy.
So, combining these two ideas, we have a daemon on the host which is
monitoring the host memory usage and the memory usage of the UMLs via
the mconsole /proc interface. When the host starts swapping, this
daemon can figure out what UMLs are occupying memory, but not using
it, and which UMLs need memory. It can then take memory from idle
UMLs by telling to allocate it and free it to the host, and give it to
others by allowing them to take back memory that they had previously
released or by giving them extra memory.
However, freeing memory back to the host requires that the memory be
released from the file that it is mmapped from. In effect, this means
being able to create a hole in the file, and there are currently no
mechanisms for doing this. There have been discussions in the past
about sys_punch or sys_fclear, since there are multiple uses for this,
but it has never been done. One major reason is that this resembles
truncate, except that this doesn't change the length of the file, and
truncate is famous for the number of complicated and subtle races that
it's involved with. Al Viro has given sys_punch a chilly reception
for this reason. So, until there's a way of releasing file-backed
memory back to the system, this plan is going to stay unimplementable.
15 Aug 2003
I'm doing a somewhat better job at keeping up with Linus these days.
I have test3 working, and I'm in the process of updating BK pools and
generating the patch. I merged in all outstanding changes from my 2.4
tree while I was at it.
I released another 2.4 patch with my accumulated changes. These were
mostly small bug fixes. I'm continuing to bang away at my 2.4 tree,
fixing some compilation warnings today.
On the conference front, I submitted an abstract to Linux-Kongress. It's been a
couple of years since I was there, and a lot has happened to UML in
the meantime. So, the abstract describes the major things that have
happened to UML in the last year or two, plus a few of the things I'm
planning for its future.
29 Jul 2003
I'm caught up with Linus now. I released the 2.6.0-test1 UML from OLS
last week. I jumped ahead instead of going release by release on the
advice of Oleg Drokin, who had already done it and found no problems.
The patch is broken, unfortunately. I tested from my BK repo, and
generated the patch from it, and didn't bother testing the patch. It
turns out I hadn't fully checked all the changes back in. I'm going
to release 2.6.0-test2 today with a working (and tested) patch.
I was at the Kernel Summit and OLS in Ottawa last week. A good time
was had by all, and all were pretty tired out by the end. The most
interesting aspect of it from my humble point of view was the interest
in UML. There was a panel of people from large companies who talked
about what large businesses wanted from Linux. Two of them (HP and
Merrill Lynch) mentioned UML.
Bdale Garbee (the HP guy) was seeing demand within HP for UML on
IA64. I talked to him later, and he said that there was some
possibility that they would do UML/IA64.
Robert Lefkowitz (Merrill) mentioned two things about UML. The first
is that IT infrastructures in the financial industry are somewhat
fragmented due to "Chinese walls" between divisions that are required
by the SEC. Large financial companies have conflicts of interest due
to the number of different things that they do. One division might be
doing business with a company, and want to keep that company happy as
a customer. Another division might be making stock recommendations to
investors and might be telling them that this company sucks and to
sell its stock. To limit the ability of one division from influencing
the other to the detriment of its customers, the SEC requires that the
divisions be structured in such a way that they don't communicate with
each other too much. These structures are called Chinese walls, and
they apparently extend to the IT infrastructure. You can't host
functions belonging to different divisions on the same systems.
Except that virtual machines on the same host are OK, which is where
UML comes in. So, having multiple functions on the same host is
legally OK, as long as they are separated by being in different
virtual machines.
The other thing he mentioned was an attempt to package a VPN client
for their employees inside UML for packaging and support convenience.
The attraction of UML is that it is a single known environment, in
contrast to the multitudes of OS versions running by the employees.
Reducing that to one environment makes installation and support much
easier. He complained that UML didn't boot off readonly media, and
said that he had told the UML maintainers about it. At this, I stuck
my little hand in the air and said, "Uhhh, I don't remember anything
about this". Talking to him later, he said he thought the person who
had done the complaining did it in such a way that it didn't appear to
be a complaint, just a "does this work?" sort of a question. My
opinion, and that of others, is that this person complained to Red
Hat.
Moving on to OLS, there was Werner Almesburger's umlsim talk, in which
he described a network simulator he built on top of UML. Russell
Coker was planning on letting people log in to SELinux UMLs during his
tutorial, but that didn't come off because of a non-UML-related
catastrophe he suffered beforehand. The clusterfs person talked about
how they're using UML for debugging. The after-dinner speaker at the
IBM dinner talked about grid computing and mentioned UML as a
possible technology that it could be based on. So, even though there
was no official UML content at either KS or OLS, there were plenty of
people talking about it.
17 Jul 2003
After much delay, and a bunch of mangled BK repositories, I decided to
get my act together with 2.5 again. I somehow ended up with all my
repos containing essentially the same stuff, with minor variations.
So, I merged them into a single tree, produced a diff, split it out,
and applied the pieces by hand to the different repos. So, my BK
situation is sane again, and I've updated UML to 2.5.70. I'll be
catching up to Linus again, and when I do, I'll see if I can get him
to take UML updates again.
22 May 2003
I released the 2.4.20-5 patch today. There's not too much in it. I
tracked down a memory leak which would eventually exhaust the /tmp
filesystem if some skas UMLs were rebooted enough times. This was due
to some /proc/mm descriptors not being closed across exec, causing
them to continue to hold down mmapped disk space. I also added chroot
and append mode options to hostfs. These are helpful for making
hostfs somewhat more secure. The chroot option lets you confine
hostfs mounts to a specified directory tree on the host and the append
option disallows any destruction of data, whether it be truncating
or deleting files.
In 2.5 news, I released the 2.5.69 patch a few days ago, and asked
Linus to pull the changes. He hasn't pulled UML in ages, so I'm not
expecting much from this. At least, it makes it clear that the old
crufty UML in the official 2.5 tree isn't my fault since I've been
keeping up with 2,5 and releasing patches on a regular basis.
12 May 2003
I just got back from Columbia (the one in South America, not the one
in New York), where I was attending the II Congreso Internacional De
Software Libre Colombia in Manizales. I was invited there to give a
couple of talks. The first, on the afternoon of the second day, was
on UML, and I talked generally about UML. It was a high-level
overview of UML and didn't get into the code at all. The second talk
was at 8 AM on the last day, and was about kernel hacking in general.
I gave an overview of how kernel hacking works, how to participate in
it, and a quick tour of the kernel source tree, with some recommended
reading in the code.
The second talk went better than the first. We (me and Gustavo, who
was translating for me) translated most of the first talk's slides
into Spanish. This was a mistake. It took time away from actually
working on the talk's substance. My Spanish is weak enough that I
looked at a couple of the slides and couldn't tell what they said, so
I got lost in my own talk a couple of times. The slides for the
kernel hacking talk the next morning were entirely in English. This
gave me more time to work on it, plus I was more comfortable during
the talk because I could understand the slides.
My talks aside, the conference was a great deal of fun. The
organizers (who included the Dean of the Engineering Faculty of the
Universidad de Manizales, where it was held) were amazingly concerned
about the comfort and happiness of the international speakers.
Among the other speakers was a Colombian Congressman who apparently
was a member of the terrorist group M-19 that eventually made a deal
with the government and became a legitimate political party. One of
the others was a Peruvian Congressman by the name of Edgar
Villanueva. You might recognize him as the author of a devastating
rebuttal to some Microsoft FUD that was circulating around the
Peruvian Congress last year.
The main room
- the university used to be a convent, and it's obvious in
this picture. This room was obviously a church at one point. I
figure it held ~750 people, and it was full for the main sessions. It
was slightly less full for my 8 AM, final day talk on kernel
hacking... Go figure.
Me and Gustavo
- Gustavo is an English Professor at the university who was
acting as my translator. We are in the little lab set aside for
decompression, talk preparation, and network access. Probably we are
working on translating the slides for my first talk.
My first slide
- Anyone who has attended any of my talks will notice some
familiarity here, except for the different language
Me and Gustavo on my second talk
- We sat next to my laptop and projector on the floor rather
than on the stage. So we are here, each with a microphone, doing the talk.
The conference was a lot more political than I'm used to. I usually
go to technical conferences with primarily technical content. This
one had a number of talks which seemed to be concerned with the
politics of Open Source, politics of getting Open Source software
accepted in government, business, academia, etc. As far as I can
remember, mine were the most techical talks, and may have been the
only technical ones. At the end, everyone seemed to consider that it
had been a great success.
24 Apr 2003
I got around to releasing 2.4.20-4 today. There were a good number of
accumulated fixes, including the RH9 fix, a couple of file timestamp
bugs, and cleanup of multi-line strings, which new gccs were
complaining about.
I also added exec logging to the tty logging facility. It turns out
that an intruder to a honeypot can arrange to run commands without
anything ever allocating a terminal. This makes those commands
invisible to tty logging.
The 2.5.67 UML was released a few days after Linus released his
2.5.67. 2.5.68 is now out, and I'll be dealing with that next week.
27 Mar 2003
After building gcc 3.2.2, it turns out that the 2.5.65 UML works OK.
Then it turned out that gdb couldn't read the new object files, so I
built a newer gdb, and everything started being fine again.
With that fixed, I pulled 2.5.66 and started looking at it. This
looks reasonably straightforward. The only tricky bit is that file
offsets are now stored in ptes. This requires that the offset bits be
arranged around the reserved bits. UML has complete control over the
pte format, so all the reserved bits are at the low end of the pte,
and the offset gets stuck in the upper end.
Some more minor fixes, and 2.5.66 boots. The patch is next, once I
get this pushed out to my public BK repository, followed by some more
pull requests for Linus to ignore.
24 Mar 2003
It's been a quiet few weeks for UML development. I've been keeping up
with Linus' 2.5 releases, but not necessarily releasing UML patches or
pushing changes to Linus. The current hold-up with 2.5.65 is that
Linus declared gcc 2.96 to be evil when frame pointers are enabled.
Of course, 2.96 is the version of gcc on my laptop, and seems to be
the only version available for RH 7.1. So, the 2.5.65 UML is on hold
until I grab a tarball of something newer (and all I can find from
gnu.org is 3.x) and build it.
I have a new 2.4.20 patch out. This fixes some minor bugs and applies
some small patches. Nothing major, but I wanted to clear the decks
before tackling some larger problems.
There was a new utilities release which fixed a uml_switch segfault.
This is highly recommended for anyone having problems with uml_switch
crashing. This is obviously very disruptive to any UML network which
relies on the switch, since the UMLs need to be rebooted after
restarting it.
27 Feb 2003
I decided to finally get a 2.4.20 UML out. So, I pushed out the
remaining 2.4.19 changes as 2.4.19-51. I updated my pools to 2.4.20,
and put out 2.4.20-1. The reason I waited so long on this one (as
opposed to almost every other 2.4 release when I released UML within a
few days of Marcelo's release) is that there was a bunch of stuff in
progress that I wanted to get settled down. The main one was skas
last fall, which is nicely stable now. I have had no complaints about
it at all in the last few months. Then, there were occasional reports
of nasty crashes (like the 'tracing myself' ones). I wanted to get
those knocked off before 2.4.20. That appears to have happened.
I'm going to wait to see whether the update has created or exposed any
new bugs and fix some problems in the utilities. When everything
looks nicely stable, I'll make the real 2.4.20 UML release.
2.4.21 looks like it's not too far away, so the 2.4.21 UML should
be released pretty soon afterwards. 2.4.20 was a special case because
of all the restructuring that was happening. Hopefully, that won't
happen again.
26 Feb 2003
I am releasing 2.5.63 today, and also sending the changes to Linus.
He takes my stuff about every 5-6 releases, so I'm not holding my
breath on him taking this right now, as I got a bunch of stuff in two
tries ago.
The big change is that the UML filesystems,
hostfs and
hppfs are in 2.5, thanks
to Petr Baudis figuring out the changes in 2.5 VFS needed to forward
port them. He did hostfs, and I used those changes to do hppfs.
hostfs seems to work reasonably, but hppfs pretty much only mounts at
this point - the shadowing from the host doesn't work yet.
19 Feb 2003
I got 2.5.62 working and released. The major pain here was the kernel
introducing its own sigprocmask (and Linus saying that the resulting
clash with libc's sigprocmask was totally my problem and he wasn't
about to rename it). After perusing the ld man page and info looking
for some way of isolating the two symbols and finding nothing, I fell
back on renaming the kernel's sigprocmask with an appropriate -D on
the compilation of kernel files. This essentially the same as Oleg's
fix, and sucks just as much.
6 Feb 2003
Linus finally took my pending UML updates. This seems to happen every
couple of months or so, at which point the changes have started
becoming fairly large. This gives me a clean slate to work from,
which is handy since the existing changes repositories had become
large enough that they started conflicting on a regular basis. I had
to merge several of them so they would apply cleanly for Linus.
I've started knocking items off my todo list. Mostly small patches
that have been sent in and have been languishing in my todo mail
folder.
David has set up a
UML
mirror on usermodelinux.org, and I've added it to the UML
download page. The LinuxVDS mirror is coming - I'm syncing it up and
will make it visible when that is finished.
31 Jan 2003
I more or less didn't do anything, UML-wise, for the last week or so.
With things accumulating, I decided to get back on the stick again. I
looked at the console driver locking because mconsole can hang when it
tries to get the configuration of a device that's on a host port. All
the things I thought of were too ugly to implement right now. There's
also the problem that it acquires a semaphore in an interrupt handler,
which leads to a panic if it sleeps there. I don't see a good clean
solution for that, either.
I did fix a few bugs though. Roger Binns found a couple of good ones,
which are now gone. A couple people noticed that early printfs don't
actually appear on the terminal until UML shuts down. An
fflush(stdout) should fix that.
I also have a bunch of tty logging changes on the way. Upi Tamminen
added timestamps and a direction flag to logging records, which let
you replay the log at its original pace, and to tell what data is
output and what data is input. He also wrote a little python script
to replay the log. I rewrote it in perl, and added flags to allow
effectively 'tail -f' of a log and to print out all data, input and
output. This shows you stuff that didn't appear on the terminal
originally, but which you might want to see anyway, like passwords.
On Tuesday, I was interviewed by the History Channel. They are doing
a show on network security, and were at
ISTS to
talk about honeypots and honeynets. So, George Bakos and I told them
all they wanted to know about UML honeypots and a bunch of other
things. It'll be interesting to see how much of this stuff makes the
final show.
On a side note - why the History Channel was doing this rather than yet
another WWII show, we decided that these guys were from the future.
That makes today history for them. It also means that UML figures
prominently enough that far in the future to send someone back to talk
about it. I guess that's some incentive to keep whacking away on it...
17 Jan 2003
I've been playing with 2.5 lately. I released 2.5.58 on Thursday,
announcing it yesterday. Linus promptly released 2.5.59, so I pulled
that and updated UML to it. I fixed some problems which were also in
2.4, so I ported them back.
While I was at it, I looked at SMP on 2.5. After some work, I got it
to build. It doesn't run. The reason appears to be the xtime locking
bug that was fixed in 2.4. So, I decided to release 2.4.19-47 so give
myself two releases to diff so I can apply that diff to 2.5.
So, 2.4.19-47 is also out, with a bunch of miscellaneous fixes and
cleanups in it.
I have accumulated some tools changes, so I'm releasing them today, as
well. The trigger for this was the 64-bit uncleanliness bugs. I
want to release the tools at the same time as the drivers so the fixes
match. This won't matter for anyone running UML on 32-bit boxes. The
other notable change was a bug in uml_moo which caused it not to write
out to the end of the output file sometimes, leaving it shorter than
it should be.
9 Jan 2003
I've been taking it easy on UML development over the last week or so.
I got busy doing some other stuff. UML is in pretty decent shape now
anyway, so a bit of a break likely won't bother anyone. People have
again started saying that they are having a hard time breaking it.
This happened last in the 2.4.19-6 to -13 range, before the skas
rework. So, it seems that I've reached that level of stability again.
I had accumulated a number of changes since -45, so I released -46
yesterday. The big news is that the network hang that a number of
people were complaining about is fixed. mistral had already found
this and diagnosed it. It just took me a while to realize that it was
the cause of the network hang.
29 Dec 2002
I finished merging the 2.4 stuff into my 2.5 tree. That all went
rather well, so now I have five more trees that Linus needs to pull.
I'm currently running diffs and bk stuff in order to prepare the
diffstat and changelogs that Linus likes in pull requests, plus
getting the 2.5.53-2 patch ready.
With 2.5 caught up to 2.4, it's time to start taking a serious look at
my todo list and start knocking things off it.
28 Dec 2002
I started merging the 2.4 changes into my 2.5 pool. I put a small fix
in early, then updated it to 2.5.53. That turned out to be pretty
much a no-op, so I released the patch. I also asked Linus (yet again)
to take my existing 2.5 changes. Doing BK stuff had started becoming
inconvenient because I had enough unmerged repositories that new
changes started crossing them, so that they couldn't be added cleanly
to an existing repo. So, I merged them all together and started using
that as my starting pool. Any repos based off that pool would have to
wait until Linus pulled the existing ones. So, having prepared to
have Linus ignore this set, I watched him promptly pull the whole lot.
He wasn't really enthusiastic about the host skas patch (which
I purposely didn't send him). His description of /proc/mm was
"crap". He's such a tactful person. This lead to a discussion of
what would be better, and it turns out he would prefer a system call
indirection system call which would allow an arbitrary system call to
be executed in the context of a particular address space. You would
pass it a file descriptor for the address space (which you'd get by
calling a get_mm system call) and a block containing the number and
arguments of the system call you want to have run in that address
space. Very general, and a neat idea.
22 Dec 2002
I made a whole bunch of small releases as I fiddled the linker scripts
and discovered a bit late on each one that I had broken a build or
produced a UML that just crashes on boot. The exception to this is
-45, which I did in response to a couple of nasty bugs being tracked
down to the point where I could either just fix them or reproduce and
fix them. Thanks to Jan Hudek and Barry Silverman for their work.
-45 also fixes a linker script make-a-UML-that-just-segfaults bug.
Hopefully, that's the last of them.
I'm beginning to feel better about UML stability now. The parade of
crash reports that followed the skas integration seems to have died
down. The two bugs fixed in -45 are a load off my mind. With 2.4
looking reasonable, I think it's time to start merging this stuff into
my 2.5 tree. That's been left alone, except for keeping up with
Linus, for a while. There are also some 2.5-specific bugs that need
fixing, and Oleg seems to be getting impatient about them.
20 Dec 2002
Today brings the release of -41. The main feature of this is that the
kernel stack size is now configurable. This is of benefit to no one
except people wanting to valgrind UML, which is only me because no one
else has my valgrind fixes yet.
Speaking of valgrind, it can now run UML. It produces reams and reams
of errors which are almost entirely noise. We're trying to figure out
how to get valgrind to be more selective about detecting only real
errors. So far, it looks like UML is hitting valgrind with code which
is too optimized, and contains code sequences which it doesn't
recognize as initializing things. It also appears to me that it is
not considering static data to be initialized.
18 Dec 2002
I released -39 and -40 yesterday. They contain mostly small bug fixes
and cleanups. The one big change is that I converted all initializers
over to C99 syntax. Then I decided to get caught up with 2.5. So, I
updated my 2.5 tree to 2.5.52, which was pretty easy, updated my BK
repositories and the patch, and sent it all out. So far, Linus hasn't
taken any of it.
15 Dec 2002
Continuing to play with valgrind. I fixed its clone problems by
ensuring that valgrind doesn't gain control of the child. This stops
the valgrind child and parent from stomping on the same data
structures and crashing each other. With that problem out of the way,
I started hitting problems with valgrind's signal delivery. Its
signal frames were bogus, which prevented UML's SIGSEGV handler from
getting fault information. UML hit a hole in valgrind's repe
handling. Then, it turns out that valgrind doesn't save and restore
signal masks across signal handlers correctly. This is where we stand
now. UML can run far enough that it panics when it's not given a
filesystem to boot on. The signal mask problem hits when UML starts
trying to do disk IO. Jeremy Fitzhardinge has been great in helping
diagnose these problems and providing fixes.
I've been having limited success in diagnosing and reproducing the
bugs that people have been reporting, so I decided to see if I could
rustle some up myself. I started up four UMLs (2 tt and 2 skas), ran
infinite kernel building loops on them, and also periodically hit them
with short ping floods and ab runs. They all ran fine for a while,
then they all ran out of tmpfs space on /tmp. This is fine for the tt
UMLs, but I didn't implement the recovery code in skas mode, so this
was fatal for them. I was hoping for bugs that I didn't already know
about though.
I ran the surviving tt UMLs this way all afternoon without any
problems. I cleaned up some code, fixed the skas stack consumption
bug, and decided to call that -38.
10 Dec 2002
I had a fairly fruitless day chasing bugs yesterday. Some I couldn't
reproduce, another I made no headway on. That one was what appears to
be memory corruption in netfilter.
I decided to dust off valgrind and see how close it is to handling
UML. clone() is a problem, but I realized something that I didn't
last time I played with valgrind, and got it handling the !CLONE_VM
case. It repaid that effort by spotting a minor buglet. It segfaults
on the CLONE_VM case still. I think it's because the two threads mess
with valgrind internal data after the call, and one messes up the
other. So, my current theory is that I need to have to child thread
immediately leave valgrind control and things will work better.
As a side-effect of that, I made UML build as a normal dynamically
linked binary when CONFIG_MODE_TT is off. This will be in the next
patch. Not a big deal, but it's one more step along the path of UML
becoming a completely normal app.
8 Dec 2002
Thanks to David Coulson telling me how to reproduce it, I fixed the
'tracing myself' bug seen under heavy network load. It turned out to
be a stack overflow caused by CONFIG_MODE_SKAS increasing the size of
a data structure which tt mode put on the stack. So, the problem
would go away by disabling CONFIG_MODE_SKAS, even though no skas code
ever ran.
I also took the opportunity to do a whole bunch of cleanup of the
uml_pt_regs struct (the bloated structure in question), and associated
macros and code. This turned a few-line fix into more than 1000 lines
of patch. This change is almost all of -36, which is now out.
7 Dec 2002
I finally got around to getting my BK repos hosted on bkbits.net. I
had set up a project there when I first got set up with BK, but had
never cloned my repos there. Larry Mcvoy had pinged me once or twice
about whether I was intending to use bkbits so he could clean it up if
not. So, I pulled all my changes over there.
I also sent everything to Linus again. Hopefully he'll pull it this
time.
Since 2.5.50 is corrupting data (and I really hope it's not my fault
:-), I'll not do anything more with 2.5 until 2.5.51. So, it's back
to bug fixing on 2.4. I've got some claims of ways of reproducing
some of the most-wanted bugs, plus a nice stack trace for another, so
I think I knock off some good bugs.
6 Dec 2002
I released -35 with a another bunch of small changes. It turned out
that signal delivery when libc didn't provide a restorer was broken.
This caused Tom's boot/root to segfault because it has an older libc
than the other filesystems. This is fixed, plus some large memory
crashes. There were also some cleanups.
I'm going to catch up with Linus now. I've been madly massaging BK to
get my repos updated. I looked at the 2.5.50 patch and spotted only a
couple of things that needed changing. One of them was the deletion
of the sys_security system call. I went to delete it from UML and was
mystified that it wasn't there. I thought I had totally missed it
until I happened to see the names of some of my files go by during a
BK pull. I took a closer look at Linus' patch, and it turned out that
other people already made those change for me. This is a major reason
that it's nice to be in the official tree. Other people start doing
your work for you.
So, I'll get 2.5.50 up and running and ask Linus to pull everything.
Then I'll start merging my recent 2.4 changes into my 2.5 pool.
4 Dec 2002
I only tested Tomcat in skas mode before I released -33. This was a
mistake because Tomcat could hang UML in tt mode. This uncovered a
fpstate size calculation bug, which is now fixed.
I also ran into a 'sleeping process nnnnn got unexpected signal : 29'
crash, which other people have been complaining about. Getting my
hands on it ensured its swift demise.
With those two bugs fixed, I decided to release -34. Not a huge
amount of change, but those are two major bugs, and it's good to be
rid of them.
2 Dec 2002
I figured out what was happening with Java inside UML. With
SA_SIGINFO set on a signal, UML didn't produce the same format stack
frame as the host. The reason this broke JVMs is that Java can induce
segfaults in the JVM. The JVM looks at the information on the signal
frame to figure out what happened. If it figures out that the fault
was caused by Java code, it converts it into a Java exception, which
the Java may or may not catch. Since UML wasn't putting information
in the stack frame in the same format as the host, the JVM was
confused about the origin of the segfault, and crashed.
I released -33 today with the Java fix, plus a bunch of smaller
fixes. I was hoping to have dealt with the 'tracing myself' crashes
that David is seeing, but I haven't figured them out yet. Oh well. I
also need to get 2.5.50 going, and get all my changes pushed out to
Linus.
25 Nov 2002
After far too much trouble, I got a 2.5.49 patch put together,
compiling, and running. This uncovered a nasty bug in skas mode which
caused signals to be blocked after returning from an interrupt. I
started tripping a check in the filesystem which was making sure that
interrupts were enabled. After a fair amount of time chasing the
problem, I tracked it down to my handler forgetting to re-enable
signals before it returned to userspace.
I also have a bunch of BK repositories all set to go to Linus.
Tomorrow, I'll probably ask him to pull them. Hopefully, he will this
time. The changes are getting rather large and it will be
convenient to have them merged into his tree.
22 Nov 2002
I finished the skas merge into 2.5 and got UML booting both in tt and
skas modes. So, 2.5 is all caught up to 2.4. Now, I have to push
this stuff out to Linus, and then he has to take it. And, I have to
merge 2.5.49 and release that.
At this point, I'm in bug-fixing and janitoring mode for the
foreseeable future. Despite my attempts to make the skas changes a
no-op for the tt code, the latest 2.4 UML is breaking in strange ways
in tt mode. So, it looks like I've introduced some bugs there, and
they will have to be rooted out.
I also pulled the 2.5.49 changes and tried them out. It turns out the
only change that affects UML is an extra argument to do_fork. Fixing
that produces a UML that works. So, look for the 2.5.49 patch soon.
21 Nov 2002
I'm busy merging the 2.4 changes into 2.5. Mostly, this is the skas
changes, but there's a fair amount of other stuff in there as well.
My 19000 line patch is down to ~3000 lines right now. I had to do a
huge merge which got rid of most of those diffs before UML would
compile again. Once I got it building, I had to fix three bugs, and
it booted again, which was nice.
With that out of the way, it looks like I can merge the rest in much
smaller chunks. This makes life much easier, since I can build and
test after each one, and if UML breaks, there is a relatively small
amount of code that I have to search for the bug.
I pushed the first working bunch of code out to my BK repository and
generated a 2.5.48 patch. When SF decides to start working again,
I'll announce it.
16 Nov 2002
I went back to work on the 2.5 tree. At this point, I'm making the
current stuff work again. This wasn't a big deal. There were some
small interface changes. The biggest problem was the initramfs
stuff. My objcopy didn't support --rename-section, so I grabbed the
latest binutils tarball, and built and installed it. Then, UML
wouldn't boot. I resorted to looking for anything helpful that Jeff
Garzik may have posted to lkml. What I found was that I needed to add
a couple of definitions to the arch Makefile to tell objcopy how to
jam a piece of arbitrary data (the initramfs image) into the kernel
binary. With that, the 2.5.46 UML boots.
While pulling 2.4.47 into my BK tree in my umlcoop UML (which failed
once with an error from deep inside BK), I decided to generate the
patch from my 2.4 tree that I'm going to have to merge into my 2.5
tree. It turns out, as best as I can figure, that my 2.5 tree is up
to 2.4.19-14. So, I have to merge the changes from -14 to -31. Ouch.
The diff is almost 19000 lines. Ouch.
15 Nov 2002
I released 2.4.19-31 today with lots of bug fixes. I'm starting to
work on the backlog of bug reports, patches, and other things that
I've accumulated since I started the skas work. I also redid a lot of
the get_config code and extended it to the ubd driver. The network
drivers still don't support it, plus the console and serial line
drivers don't support plugging and unplugging devices at run time.
I also need to support listing of all devices of a particular type,
which is needed for people who want to find a spare slot into which to
plug a new device.
-31 doesn't compile with CONFIG_MODE_SKAS disabled. It was a silly
oversight - I put some skas-specific code outside CONFIG_MODE SKAS.
I'm starting work on 2.5 again. The last UML release was 2.5.44.
Linus is up to 2.5.47. I'm going to take the existing code up to that
(or .48 if Linus releases that before I'm done). Then comes the job
of merging all the changes I've made in the 2.4 pool. The largest
piece of this is the skas stuff, but there have been a lot of bug
fixes, cleanups, features, and other changes.
13 Nov 2002
I finally figured out the FP problem that was preventing RH7.2 from
booting on -29. I wasn't initializing the FP state correctly in new
processes. Copying in a known good set of values fixed that.
I also fixed the segments problem cleanly. The host skas3 patch
included a copy-segments /proc/mm operation, which is needed to copy
arch-specific address space information from one address space to
another. This wasn't used before, but is now in order to copy x86
segment information between address spaces.
With that, plus a bit of code cleanup, I released -30. At this point,
it's time to settle things down, fix bugs, clean code, and get 2.5 up
to date again. Basically, I'd like to spend a good amount of time on
maintenance rather than new code. We'll see how well I resist the
call of new functionality...
11 Nov 2002
I'm releasing -29 and another version of the host patch today. The
big news here is that, in skas mode, UML no longer creates one host
process for each UML process. That used to be done in order to create
new address spaces on the host. Since only one of those processes
could be running at any given time (in a UP UML), they were a waste of
kernel memory.
Now, I've added /proc/mm, which provides a way to deal with address
spaces independently of processes. Opening it creates a new, empty
address space, and returns a file descriptor which can be used to
manipulate it. Closing the descriptor frees the address space if no
process is running in it. The address space can be populated by
writing to the descriptor. You write a request to it, which can cause
an mmap, munmap, or mprotect to happen. The contents of the request
are basically the arguments to the corresponding system call.
This allows a process, i.e. the UML kernel process, to have handles to
many address spaces without processes running in them. When UML
creates a new process, it creates a new address space for it. When
that process is scheduled, the UML userspace process on the host is
switched into that address space with PTRACE_SWITCH_MM. That causes
it to jump from its current address space into the new one. That,
plus a full register restore, is now a UML context switch.
There are some glitches with it. Floating point is broken. This
doesn't affect older distrubutions such as Debian potato, but it
breaks newer ones, such as RH7.2. The problem is that, while I do a
full save and restore of the floating-point registers across a context
switch, that's not good enough. What I need to do is tell the FPU
(via fxsave) to dump its state out to memory. That will give UML the
real FP register state, which would be restored when that process is
next scheduled.
With these problems fixed, I've reached the level of host support I've
wanted for a while. The number of processes on the host is close to
the absolute minimum. This causes UML to consume fewer host
resources. It also seems to be faster. I ran some kernel builds, and
it now takes 8:22. I didn't run tt or skas2 builds, but if my
previous results still hold, which they should, that's 45 seconds
faster than skas2, which came in at 9:07. That brings the skas3
kernel build time to 60% of the tt build time.
The one major thing that I could still ask from the host is some way to
merge the current kernel and userspace host processes into one. This
would involve some way for the kernel process to switch itself to the
userspace address space, with a register switch at the same time. It
would also require a mechanism to switch back (address space plus
registers) on any sort of signal. This essentially would mean it was
ptracing itself, plus intercepting its own signals. It's not clear to
me how to do that, so I'll just let this sit for a while.
There are a couple minor host changes which I'll probably add in at
some point. ptrace currently requires that system call intercepting
processes see both the start and end of the child's system calls.
This causes four host context switches per UML system call. What I
would like is to just see the start of a system call, at which point
UML would read it out, run it, and stick the result back in the
child. It would be told just to return immediately to userspace.
This would eliminate two of those context switches, and bring it down
to the theoretical minimum. This should also noticably improve
performance.
The second minor thing I am planning is some way of filling in the
start and end addresses of the command line and environment of new
address spaces. Those values would be the same as the UML values.
This would have the nice effect of making ps on the host show the same
process name for the UML userspace process as ps inside UML. So, that
one host process would change its name according to what is currently
running inside UML.
I got rid of the ugly asm that I used to set up %gs correctly. There
is support in the host patch to copy segments from one address space
to another, but UML doesn't currently use it. This is
straightforward, and should be the correct fix for the segment
problems I was having last week.
There were a bunch of other changes in this patch, including
/proc/mm support, since I tested it in UML before porting it to the host
I fixed the segfault caused by querying the configuration of a device
that had never been opened
UML now compiles with CONFIG_MODE_SKAS off.
There was a fair bit of code cleanup.
Fixed the behavior of the network driver when it gets an error.
5 Nov 2002
After much wailing, gnashing of teeth, rending of garments, and asm
nastiness, I fixed the problems with my RH 7.2 filesystem not booting
in skas mode. The problem is with the segment register gs, which is
used somehow by newer pthreads libraries. UML was ptracing the
correct value into new processes, but it was vanishing immediately for
reasons that I couldn't figure out for a while.
It eventually dawned on me that it would work if it was inherited from
the parent process. In tt mode, the parent process is the host
process of the parent, so that works automatically. In skas mode, the
parent process is the kernel process, so there is no relationship
between the host processes that hold UML parent and child processes.
With this level of understanding, I went about setting the value of gs
in the kernel process temporarily to the correct value so it could be
inherited by the child. This consistently segfaulted, which I didn't
understand till this morning.
It turns out that loading a value into a segment register will fail if
the corresponding segment hasn't been loaded with sane data. Now,
modify_ldt is there, but it operates on the userspace process, not the
kernel process. So, I added a kludge to make modify_ldt also operate
on the kernel process. This allows the segment register to be
assigned and then inherited properly by the child.
After staring at the i386 code, I have since realized why the value of
gs can't be ptraced in. It is added to the thread structure properly,
so if you write it in and immediately read it back out, you will get
the expected result. However, in the i386 switch_to, it zeros out fs
and gs if they don't have good segments already. I'm not sure exactly
why they don't, but this looks like the reason ptrace isn't working
the way I expect.
So, with that fix in, plus a bunch of others, -27 is out. If you want
to use skas mode with it, you'll need the
host skas2
patch. The first patch won't work because I moved the new ptrace
operation numbers so that -27 won't see it even if it is applied to
the host.
3 Nov 2002
-26 is out. This contains a few bugs fixes which are enough to get my
RH 7.2 filesystem to boot. It needed a working modify_ldt and a
signal delivery fix. I started trying to run UML inside itself in
skas mode, and that turned up a timer bug. With those bugs fixed, RH
7.2 boots, but a few of the daemons are running continuously. This
needs fixing as well.
To take advantage of this, you'll need the second host patch, which I
haven't released yet. The one I'm running on my laptop has
PTRACE_LDT in it. I'm going to add PTRACE_JOIN_MM to it before
releasing it. I played with that a bit today, with a little test
program that is made to switch address spaces. It seems fine, and
I'll port it to i386 soon. I also need to port PTRACE_LDT into
UML since that's only in my i386 pool at the moment.
I haven't talked about where I'm going with the host support, so this
would be a good time to do it. PTRACE_JOIN_MM is a stepping stone
towards greater things. Currently, you give it a pid, and it will
locate that process and make the ptraced child join its address
space. The next step is to eliminate that pid, since it's only used
to locate an address space.
I'm going to make address spaces first class objects which are visible
from user space. This will probably take the form of /proc/mm, which
when opened, gives the caller a file descriptor whose underlying
object is a brand-new, empty address space. Then, PTRACE_JOIN_MM can
dispense with the pid, and use that file descriptor instead.
What this gives us is the ability to have an address space without a
process running in it. In turn, this lets us eliminate the host
processes which each hold a UML process userspace. In its place, we
will have one host process per UML processor to hold the userspace
contexts. The one host process per UML processor kernel context will
remain. So, the number of host processes will go from
num_UML_processors + num_UML_processes to 2 * num_UML_processors,
which is much smaller. The userspace context processes will get
bumped from address space to address space as needed.
That much is easy. It leaves us with each UML processor having two
host processes, one for its kernel context and one for its userspace
context. The next step will be to merge those, and have one process
per processor bouncing between kernel and userspace address spaces.
This is harder, because it implies these processes ptracing
themselves, with an automatic address space switch every time there's
a ptrace event which causes a switch to kernel space.
For system calls, this seems straightforward. The system call tracing
code can do the address space switch, and replace the registers to be
restored on return to userspace. Signals are harder because that code
is deeper down the stack, so more code would have to be fiddled in
order to make that work.
Assuming that's doaable, that's the ultimate goal - one host process
per UML processor, with one host address space per UML process. This
would eliminate a bunch of the ptrace additions, which would have
served temporarily as scaffolding. PTRACE_{MMAP,MUNMAP,MPROTECT} and
PTRACE_LDT would become operations on the /proc/mm file
descriptors. PTRACE_SIGPENDING becomes unnecessary since that's needed
to avoid races when switching between processes. PTRACE_FAULTINFO
would still be needed, unfortunately, since PTRACE_GETREGS doesn't
give it to you.
2 Nov 2002
I released -24 and -25 over the last few days. They are mostly bug
fixes to skas mode. The network now runs reasonably, thanks to fixes
to the checksumming code, although I do have sshd's hanging around
after copies. There is now thread support, which allows me to run
kernel builds - make uses vfork, which is a cheezy form of threading.
So, I did some kernel build runs to see what performance difference
there is between tt and skas modes. Two runs each in skas and tt
modes produce times of 9:07 and 13:50 respectively, identical to the
second in both cases. This makes the skas time 65% of the tt time.
For some micro-benchmarking action, I ran a loop of 100000 calls to
getpid. In tt mode, it takes 15 sec; in skas mode, it takes 7.
There are still some improvements to be had. Currently, skas mode
does a system call with four host context switches, while tt mode does
one with four host context switches and a host signal delivery and
return. The speedup comes from losing that signal delivery, plus
maybe the context switches are faster, due to the UML process address
spaces being smaller because they don't contain the kernel. Two of
those four context switches can be eliminated, which should cut the
system call overhead down by a factor of two from where it is currently.
In the continuing quest to get my RH filesystem to boot in skas mode,
I fixed modify_ldt. This is another address-space-changing function,
which, before I fixed it, was modifying the kernel address space,
rather than the process'. So this required another ptrace kludge on
the host, which when implemented, made modify_ldt work better. It
still doesn't boot. The current hang-up is something called getkey,
which I've never heard of, has no documentation that I can find, and
has no interesting strings in its binary. It is sitting in a poll
forever, and I haven't figured out why yet.
30 Oct 2002
I released -23 last night. UML now runs in skas mode on the host. I
ported the host ptrace patch from UML to i386 and have been running it
on my laptop for the last few days. It is available
here
.
Once skas mode is working well, it's time for some stabilizing. With
highmem, SMP, and skas support going in over the last couple of
months, I'm going to concentrate on killing bugs for a while. Marcelo
has started the 2.4.20-rc series, so I'm thinking that I will start
new UML development projects after 2.4.20 is released, and concentrate
on stability until then.
27 Oct 2002
skas mode works now. The host has to be a recent UML, and the guest
needs to be built with CONFIG_MODE_SKAS enabled and CONFIG_NEST_LEVEL
set to 1. See the story on
usermodelinux.org
about the 2.4.19-21 release. That contains a good description of what
skas mode is and what the benefits are.
I released -22 today with more fixes and cleanups. UML now builds and
runs with either mode configured out. I fixed the SMP bu ild, although
I punted on SMP on skas mode for the time being. This invovled a fair
bit of code cleanup and movement.
I pursuaded UML to build as a normal dynamically linked binary. This
took some surgery on the linker script which I didn't include in this
patch. The point of that exercise was to see if
valgrind
would work on UML now. It starts to, but blows up on the first call
to clone, which happens fairly early. After some correspondence with
Julian Seward, the valgrind author, it appears that this is easy to
fix. So, we may soon have valgrind working on the kernel, which is
timely considering that we are entering the stabilization phase of 2.6.
24 Oct 2002
Most of the last week was spent merging the skas work done back in
September into my main pool. I created a couple of pools on the side
so that I didn't have to stall the main line of development. Those
two pools are now merged into my main pool and are in CVS and the
patch. I released 2.4.19-18 yesterday with everything finally
merged. The skas changes are present, but latent for now. You can
enable CONFIG_MODE_SKAS (it's hardwired to on currently anyway) and
the code will be compiled in, but you can't run it. -18 had some
build problems. I forgot to update a clean rule, with the effect that
there's a binary in the patch (rm arch/um/util/mk_constants if you
want to build that patch). The link will fail if you enable
CONFIG_HIGHMEM.
I released -19 with those bugs fixed and with some code movement, and
detection of the host support needed for skas mode. This gives me a
clean platform from which to start debugging the skas code, and
everyone else a cleanly building patch.
14 Oct 2002
I spent the last few days debugging the merge of the 2.4 SMP support
into my 2.5 pool. That was more painful than I expected. The
actual debugging took longer than the 2.4 debugging, notwithstanding
the fact that I had the benefit of that earlier debugging. The SMP
infrastructure had changed in 2.5, and I didn't get UML into userspace
until I stared at the i386 SMP boot process and made the UML boot do
the same things at the same time.
I had to change the locking in the ubd driver. I started with the
request queue lock being the same as the ubd device lock, which
deadlocks unfixably when a disk is added to the system with mconsole.
ubd_config holds the ubd_lock, calls add_disk, which does IO to read
the partition table, which tried to grab the ubd_lock again. I added
a ubd_io_lock, which pretty much restored the 2.4 situation, which
protects IO with the global io_request_lock.
On 2.4, I had all interrupts handled on CPU 0. This turns out to be
wrong - all processors have to have timers in order to do their local
process accounting. However, only one processor, CPU 0 in the case of
UML, actually calls the timer IRQ. This matters on 2.5; it doesn't
seem to on 2.4. I also found an unfixable race with the timer
interrupt caused by UML never blocking SIGALRM and SIGVTALRM and
instead relying on flags in order to call into the kernel when it
shouldn't. I decided to just get rid of those flags and treat the
timer interrupts like any other signal.
The end result is a bunch of bug fixes and cleanups which need to be
carried back to 2.4.
I'm currently merging 2.5.42, and watching an SMP 2.5.42 do a -j5
kernel build. So far, so good. However, there are some strange
crashes which I haven't figured out yet. They were pretty
reproducable for a while, then they seemed to disappear and I got 3
-j5 kernel builds in a row.
The evil thought of the day is to port UML back into the kernel. The
internal kernel interfaces can be thought of as another OS, which
would make this an OS port of UML. Why dump UML back in the kernel
after I've spent all this time pulling it out into userspace? There
are a few reasons that come to mind
It's whacked, therefore it appeals to me and must be done
It could perform better since it has access to the full host kernel
and is not restricted to the system call interface. This is another
way of looking at UML performance and could lead to better ideas on
making normal userspace UML perform better.
It would go in the direction of allowing Linus to partition a machine
and run separate OSes on the different partitions. The host could
hand devices to the UML, which would access them using the normal
hardware drivers, something it can't do in userspace. Over time, the
"host" kernel could be trimmed down to the point where it's nothing
but a little executive which starts the partition kernels and splits
the hardware between them. Then the "guests" would be the real
machine OSes.
8 Oct 2002
Several 2.4.19 patches later, the SMP audit is done. UML/SMP seems to
be OK. My test before releasing -12 was a -j5 kernel build on a four
processor UML. That went fine. mistral is reporting a hang, but I
want to see it myself or else I'll consider that he's just making it
up :-)
I released the 2.5.41 patch today, and pushed everything out to Linus
again. I accidentally left out port numbers on the URLs I gave Linus,
so I quickly put a bkd on port 80 of my UML so that he would be able
to pull everything.
The next task is to merge the SMP stuff from 2.4 into 2.5. That's the
reason I cleared the decks on 2.5 today.
4 Oct 2002
I've started the SMP audit. The first step was to go through the code
marking all global data that took longer than .5 seconds to tell that it
didn't need to be locked. The next step will be to go through those and
add locking where necessary. I've done some of this, also adding comments
where locking is not needed.
With this, I released 2.4.19-9. I also turned off CONFIG_UML_NET_PCAP
in defconfig since the default build would break on any system that
doesn't have libpcap installed.
1 Oct 2002
Some Boston University students seem to have some intellectual
integrity problems. Apparently some class was assigned the task of
describing the differences between the virtualization approaches of
UML and VMWare. So, what do these fine students do in response? They
send me mail such as
Would you like to tell me that what are the fundamental differences in the way
virtualizition is performed in UML vs. VMware?
I am eager to know about it.
and
I'm hoping you can help me with this. I'm doing a small research project and
am trying to figure out some of the fundamental differences in the way
virtualization is performed in User-mode Linux vs. VMware.
Ummm, right.
In actual news, I released 2.4.19-8 today. I removed the limit on the
number of network interfaces a UML can have. In related work, I
cleaned up and simplified the network transport interface. This
release contains a good number of small bug fixes.
Linus merged my changes into 2.5.40, so that's up-to-date with respect
to generic kernel changes and highmem. I still have a bunch of bug
fixes waiting. I'll be pushing them plus the networking changes.
30 Sep 2002
Highmem is now working in 2.5. UML is also updated to 2.5.39, so I
sent both the 2.5.39 and highmem updates to Linus. I released the
2.5.39 patch as well.
I now have a lot of pent-up small stuff to deal with. I'll clean out
some of that, and then get on with getting SMP working.
23 Sep 2002
I finally released the 2.4.19 UML after fixing some final build bugs.
With that out of the way, I can start working on more intrusive stuff.
In the intrusive stuff category, I got highmem support working. So,
you can specify any amount of memory you want, up to 4G, and the stuff
that can't be mapped directly will become highmem.
I released 2.4.19-6 with highmem and fixes for a couple of crashes.
One of them was a subtle timer bug that shows up with an idle UML on a
loaded host, namely umlcoop.org.
I think I've got BitKeeper figured out. I'm using it for 2.5
development, and have got UML updated to 2.5.38, which patch I'm
releasing now. I have to update the repositories on umlcoop.org so
other people can pull from them. After that, the next task will be to
get my 2.4 stuff into it.
13 Sep 2002
Lots of people are happy about UML finally getting into to 2.5. I've
gotten email from all over the place. It was also fairly newsworthy,
apparently. It was first picked up by
kerneltrap. A pointer to that story was submitted anonymously to
usermodelinux.org,
which I posted, notwithstanding the fact that I had just posted a
story of my own about it. An almost identical submission was made to
Slashdot,
who ran it. This caused more traffic to
usermodelinux.org
than it had ever seen before. To my knowledge, this is the first time
that a UML had been slashdotted. There were some hiccups, but it
seems to have fared reasonably well. It helped that it wasn't a
full-bore Slashdotting.
It was also picked up tangentially by LWN.
Linux Today
ran a story that just pointed off to kerneltrap.
12 Sep 2002
I released 2.4.19-3 yesterday and announced it today. I also released
2.5.34, and sent the patch to Linus.
I think I'm getting close to deciding that 2.4.19 is done. The
problems that have cropped up that I don't have fixes for don't seem
to be as serious as they first looked. The problems that I do have
fixes for will be fixed, of course.
The one remaining fix that I have is the tracing thread crash in
kmalloc. I'll put that in 2.4.19-4 and see how that looks as the
official 2.4.19 UML.
Late breaking news : Linus finally merged UML. It will be in 2.5.35.
9 Sep 2002
2.5.34 is out and UML isn't in it.
I've been playing with a UML on a colocated server bought by Bill
Stearns and a bunch of friends, and set up by Bill over the weekend.
Everyone who contributed gets a private UML with its own IP. I've
turned mine into another mirror of the
UML downloads and
web site. I also put CVS on it with a view towards moving my CVS
off SourceForge. If I can get the server side of BitKeeper (which
I've downloaded, but not looked at to see whether I got only the
client side), I might also run my BitKeeper pools from there.
After this, I had an evil thought. A root filesystem with
Apache
PHP
mod_perl
MySQL or Postgresql
CVS
Mailman
A web site, or other documentation, explaining what's there and how to use it
installed would make a fine project hosting platform. Couple it with
a host and an administration that can keep it running, and you have a
good mini-SourceForge. I would certainly prefer that to SF, mainly
because of the control and flexibility I get.
In other news, I spent today fiddling the web site. A lot of time was
spent cleaning up the XML. I fixed a bunch of mistakes and got rid of
some obsolete information.
6 Sep 2002
I've spent the last few days catching up with bug fixing and releases.
I rolled a bunch of fixes in and released 2.4.19-2. I also made a
tools release with a few changes, chiefly fixing uml_moo so that it
spits out a sparse file and cleaning it up so that it's much more
understandable.
I also got 2.5.33 going. None too soon, since the natives were
starting to get restless. James McMechan and Mike Anderson both
popped up on uml-devel with the changes needed to get UML up to
2.5.33. I'm sending UML to Linus again. We'll see how it goes.
2 Sep 2002
Today's problem is SA_RESTORER. This was a hidden same-address-space
dependency. UML signal frames are basically copies of host signal
frames with all of the signal-specific information replaced. Since
the libc in UML provides a restorer that's inside UML, UML processes
use it since that information is not changed when UML constructs its
own signal frames. This works fine as long as UML and the process are
in the same address space. When they aren't, and the process' libs
doesn't provide a restorer, the process will segfault trying to call
UML's restorer.
The fix is to get a frame from the host which has the kernel's
built-in restorer, which you get by turning off SA_RESTORER in
sigaction. However, this is easier said than done, since the
sigaction entry point provided by libc specifically provides a
restorer and disables any attempt so say that there isn't. So, I
ended up calling sigaction by going through the generic syscall()
entry point rather than libc's sigaction. Doing this, plus using the
old sigaction structure that the kernel uses got me a signal frame
with a built-in restorer.
Theoretically, this should get the new UML up to a login prompt. The
signal frame code that I changed in the host UML also needs to be
applied to the guest UML. And I decided to do that as part of the
merge of the two pools. And that is going to wait a bit.
I have a bunch of bug fixes to get out, plus I need to catch up to
Linus, plus I need to get 2.4.19 out. So, I'm going to take care of
all that, and then get on with merging the re-architecting into the
main UML pool.
30 Aug 2002
Today was Signal Delivery Day. That took me a while to get right,
mostly because it was a nice day and I felt like doing stuff outside.
That done, a bug in fork reared its ugly head. It turned out that
fork wasn't returning 0 in the child, making it believe that it was
the parent. Since init is doing the double fork trick, where it
forks, the child forks and exits, and the grandchild execs whatever
needs execing, this meant that all children just exited. init
understandably got rather upset with this.
exec then crapped out because of a buglet in strnlen_user. I fixed
that, plus a few instances of another bug that I happened to notice.
Now, I get a getty running. It hangs in sigsuspend. I think that's
because it's sleeping for a few seconds and I haven't hooked in the
timer yet.
29 Aug 2002
Decent progress today. Last night's infinite segfault loop turned out
to be caused by clear_user not really clearing anything. So, init's
bss, which was supposed to be zeroed, contained garbage, and it
crapped out when it tried to dereference some of it.
Fixing that and one or two other user access bugs made things work a
lot better. init now runs, and starts firing off the rc scripts. The
current hold-up is signal delivery. init won't boot the system unless
it sees SIGCHLDs from exited scripts. There are only minor problems
here, so this shouldn't be a big deal.
28 Aug 2002
Today's installment of As the Codebase Churns stars the user access
macros. I implemented the rest of them when it turned out init was
getting to the first open, but bombing on a copy_from_user. That took
little time, but I spent the rest of the day chasing bugs in them.
The problem is that when you mess them up, the kernel doesn't crap out
right there. It bombs at some later point, in an apparently unrelated
way. For example, I was chasing an infinite segfault loop. More or
less by accident, I discovered that init was being started with an sp
of zero. That turned out to be because a function which sets up the
initial stack for an exec was silently failing. And that was failing
because of a bug in strnlen_user.
I *always* get strnlen_user wrong because it's defined differently
from the libc version. The libc strnlen normally returns
strlen(str). The kernel version returns strlen(str) + 1. Believe it
or not, this makes a big difference in how well (or if) your kernel
boots.
After chasing this and a few other entertaining bugs, init is opening
files, making system calls, and all those good things. Unfortunately,
it also goes into an infinite segfault loop on a rediculous address.
This is something to chase in the morn.
27 Aug 2002
I fixed last night's hang by having UML collect a representative set
of registers, including a set of good segment register values, from a
subprocess. This is used as the starting point for new processes,
rather than the array of zeros I was using before.
With this fix in place, init gets to the point of starting to make
system calls. The new system call handling required redefining
the pt_regs structure, which required a fair amount of hacking and
slashing before it would compile again.
It now seems to be doing system calls OK, although with some
flakiness. I've currently got it up to system call number 6.
26 Aug 2002
The Great Redesign continues.
The major task of the day was redoing the copy_user macros. When the
kernel is moved from the process address space to its own address
space, the macros which copy data back and forth between userspace and
kernelspace get more complicated. Before, with kernel and process
sharing the address space, data can just be copied back and forth,
taking care to make sure that faults are handled properly, since the
userspace address may be bogus, or it may have been swapped out.
That's no longer possible when they don't share an address space, so
what you have to do is do a process virtual to physical address
mapping, and then copy the data in or out of the physical memory.
When the page isn't present, there won't be a mapping for it, and you
have to fault it in to the process address space. Then, the mapping
will be created and the data is accessible.
With copy_to_user and clear_user in place, it's possible to exec init
and enter userspace. I've got to the point where it has faulted in a
few pages. init then spins because I gave it a set of registers which
is all zeros, except for the ip and sp. For the normal registers,
this is fine, but things work very badly when the segment registers
are zero. In this case, it's spinning on the first data reference
because it has a ds of zero.
In other news, it turns out that I pretty much have to totally work
inside UML. Since I'm running UML inside another UML with the kernel
pool on a hostfs mount, and because hostfs doesn't consider that files
can change underneath it, I have to edit and compile through hostfs,
rather than doing that on the host. I finally installed emacs in UML
after getting sick of @!$!@%$ vi, and I'm fairly impressed by how
quick it is. I can see a little bit of interactive slowness
sometimes, but it pretty much feels the same as emacs on the host.
Builds are another question. They are noticably slower than on the
host. This might be due to hostfs and its synchronous IO, but I'm not
sure about that.
25 Aug 2002
I'm in the throes of redoing parts of UML so that the kernel lives in
its own address space. There would be one thread per processor
running in this address space. Every UML user process would have a
separate host process. Transfers into the kernel therefore cause a
context switch from the user process to its kernel process.
It's being done this way because of Ingo's revelation on LKML that
context switches are much faster than signal deliveries. Previously,
I was thinking that the best performance would come from intercepting
system calls without context switching. I was planning on having the
kernel optionally be put in its own address space, but was thinking
that there would be a performance penalty for doing that. Now, it
seems that this is totally the best way to go.
To support this, I've added a bunch of new stuff to the host's ptrace:
PTRACE_FAULTINFO - get the fault type and address of the child's most
recent segfault. This is needed for page fault handling.
PTRACE_SIGPENDING - get the child's pending signal mask. This will be
used to work around a race between a child executing a system call and
a SIGIO being queued to it. If they happen at the same time, the
system call cause control to be transferred into the kernel, leaving
the SIGIO pending on the child. If the kernel then switches to
another process, that IO notification will be lost for an indefinite
period of time, which can UML to hang.
PTRACE_MMAP, PTRACE_MUNMAP, and PTRACE_MPROTECT - these manipulate the
child's address space.
PTRACE_CLONE - not added yet, but this will cause the child to call
clone. The reason for this is a little subtle. CLONE_VM threads in
UML need to be CLONE_VM on the host as well. So, the kernel process
needs to be able to create a clone of one of its children.
When a user process is running, the kernel process will be in a loop
calling wait and ptrace in order to intercept system calls. When a
process system call bumps the kernel process out of wait, it will read
the system call and execute it itself. When it returns, it will
switch back to the user process. So, a UML system call will have an
overhead of two context switches rather than the current four context
switches, one signal delivery, and one signal return.
Kernel threads don't have an associated host process.
Context switches are done with a longjmp from one kernel stack to
another.
Traps into the kernel are done in the same way as system calls. The
interrupt to the user process will be intercepted and cancelled by the
kernel process, which will execute the appropriate handler. The
performance increase will be about the same as for system calls.
The benefits of this are many:
UML should be noticably faster.
jail mode is now automatic, so kernel memory protection is always on,
and is very much faster than it is currently.
processes now have the normal full 3G of address space. This makes
honeypot mode automatic (and much more believable since the UML kernel
will no longer be visible to processes). Applications which require a
lot of address space and which may have bombed out on UML as a result
will now run.
The UML kernel now has a full 3G of address space. This makes it
possible for UML to conveniently have lots of physical and/or virtual
memory since it can all be mapped at the same time.
UML can now be a normal process, rather than the strange, statically
linked, oddly loading process it is now. UML can be debugged with
'gdb linux', rather than the ptrace proxy arrangement it uses now.
gprof and gcov should work much better as well.
Many, many kludges just vanish. The shared remapping of kernel text
and data is gone, as is switcheroo. Context switching is dead simple,
and obviously race-free. Signal handling is much simpler, since there
is no more special handling of the startup of a new process. As
already mentioned, the ptrace proxy and all its nastiness goes away.
Since this requires changes in the host, and the world may not patch
all of its kernels immediately, I'm planning for UML to be dual-mode
for the foreseeable future. It will detect the support and use it if
it's there. Otherwise, it will fall back to the traditional tracing
thread. In the code, I'm planning on separating the code that
supports the two modes into separate subdirectories. This provides a
reasonable way of splitting out code that's been somewhat messy for a
while, and separates the two pieces of code from each other. It will
also make it easy to configure a single-mode UML, in case you know
that you will be running in one mode, and you don't want the code for
the other mode compiled in.
So, what's the current status? I've added the new PTRACE_* options,
except for PTRACE_CLONE to the kernel. The arch-specific support is
only in UML right now. I'll add it to i386 when UML is up and running
on it. As a result, I'm debugging the new UML inside an old UML with
the ptrace additions. When it's working, I'll un-nest UML, fix the
i386 kernel, and see how much faster it is.
The new UML has gotten to the point of starting to fault in init.
Right now, exec is setting up init's stack and some of its data,
forcing a few of its pages to be faulted. It has not yet entered
userspace. It has created the usual collection of kernel threads,
initialized them, and switched between them. So, the in-kernel
portions of this code appear to be working well.
22 Aug 2002
I decided to flush out all the pending changes I had, so I also
released new versions of the test suite and tools yesterday.
With 2.4.19 more or less out of the way, I decided to start into a
comparatively large and risky project. So, I decided to start working
on the changes needed to move the UML kernel into its own address
space. This was discussed on the kernel list a few weeks ago. As the
things stand now, this can't be done without some extra support from
the host. I'm implementing this stuff in UML now, and will move it
into the x86 kernel later.
21 Aug 2002
Not too much to report recently. I released -53 today, which contains
a few small fixes. 'jail' mode works better now with a couple of
crashes fixed. I also cleaned up the ubd driver's error reporting.
With no complaints about the larger changes that I made recently, I
think it's time to release the 2.4.19 UML. I was holding off on this
until those changes got banged on some and I was happy that they
didn't break anything. This now seems to be the case.
OK, I updated UML to 2.4.19 and released the patch. The full release
will come after I've had a chance to bang on it some.
12 Aug 2002
Linus released 2.4.31 yesterday, again without UML. However, in stark
contrast to earlier releases, the two patches to generic code were
applied. This gives me some hope that Linus will merge the main piece
of UML at some point.
I tried getting UML running on my 2.2 box on Friday before leaving for
a camping trip. The intent was that I'd leave some stress tests
running on UML for a couple of days and see what new and interesting
bugs popped up. It turned out that the bugs popped up in the process
of building and running UML. I abandoned that plan until I fixed the
problems I saw.
I knocked those bugs off today, and released the results in -52. I'll
be trying the 2.2 thing again.
Then, it'll be off to the 2.5.32 races, with yet another patch going
to Linus.
I've got 2.4.19 sitting in the background. I want the current UML to
get banged on a bit before I upgrade it to 2.4.19.
9 Aug 2002
UML development has been cleanup lately. I found and fixed a "I'm
tracing myself and I can't get out" panic. This induced me to
understand the timer flags, which let me simplify them enough that I
can understand what's going on without lots of deep thinking.
Having done that, I merged sig_handler and irq_handler_common, which
were basically the same, except for one line. Then, I merged in
syscall_handler, so I went from three copies of the kernel entry and
exit code to one.
I also discovered a few more bugs in my stress testing. These are
fixed, along with some cleanups caused by me trying to build UML on
2.2.
So, this is all released in -51.
5 Aug 2002
There is hope on the getting UML into the 2.5 tree front. Linus has
merged the changes to generic code, and most of the linkage.h patch.
The stringification I sent him apparently breaks on some compilers, so
he backed that out. I copied the syntax I used from some gcc
documentation, which I strangely can't find any more. What I do find
is somewhat different.
However, he hasn't merged the main body of UML yet. So, we'll see
what's there when 2.5.31 comes out.
There's been some discussion on the kernel list about making UML
faster. Alan floated an idea for making jail mode faster. I pointed
out some flaws in it. I floated my ideas about making address spaces
accessible from userspace, and Alan dinged them in turn. Read all
about it over at
usermodelinux.org.
In a separate development, Ingo pointed out that process context
switches are much faster than signal deliveries. This opens up the
possibility of implementing jail mode by having the tracng thread also
run the kernel side of things. This will give us a fast jail mode and
speed up system calls. The only thing that's missing is the ability
of the kernel process to change the address spaces of the other processes.
In development news, I discovered a race which produced the dreaded
"I'm tracing myself and I can't get out". This prompted a cleanup of
the timer code. 2.4.19 having just come out, I was nervous about
releasing the 2.4.19 UML with something with potentially subtle
consequences like that. So, I've been testing more heavily than
usual. I added a test to the test suite which runs the rest of the
suite with jail mode enabled. The reason for that is that jail is
sensitive to bugs in how the timer is handled.
2 Aug 2002
Linus released 2.5.30 and UML again got dropped on the floor. So, I'm
in for another round of patch generation.
2.5.30 was fairly easy. There were some block layer changes which
broke the compile, but it was simple enough to figure out what had
happened and fix the ubd driver.
30 Jul 2002
-48 is out. I reproduced the crash that mistral saw when he killed
console xterms. I turned on slab poisoning and it became 100%
reproducable. It took me a surprisingly large number of attempts to
fix it. So, that fix is in, plus the fix for hostfs compilation
failure that everyone seemed to think was a typo.
28 Jul 2002
I announced UML 2.5.29 to the kernel list and sent UML to Linus again
for him to ignore. I had redone include/linux/linkage.h in order to
clean it up and remove it from the list of generic files that UML
changes. Keith Owens noticed that and suggested an improvement to it,
which I will add the next time I send UML in.
I got hppfs to the point of allowing an outside script to generate
dynamic /proc content and to filter the real /proc file. That's the
bulk of the functionality that's needed. Then, it will be time to
flesh out the remaining file and inode operations so that everything
works as expected.
27 Jul 2002
Linus released 2.5.29 last night and guess what's not in it. Right.
The patch itself contains no particular arch changes, so it looks
fairly simple. I'll release the UML 2.5.29 today probably. I
released 2.4.18-46 today. The main feature is start of hppfs, the
Honeypot procfs. It's far from done, but works enough to allow proc
files inside UML to be replaced with versions on the host. This by
itself is enough to make a fairly convincing honeypot. I still have
to complete the file operations, so that everything works as
expected. I also have to add support for allowing something on the
host to generate files, either from scratch or from the contents of
the UML proc file.
26 Jul 2002
I sent the latest UML to Linus. We'll see if it gets in 2.5.29. If
not, I'll just keep sending it until he gets sick of the bandwidth
that it's consuming...
In actual work, I started implementing the Honeypot procfs
filesystem. This is designed to cut the heart out of the problem of
making a UML honeypot look like a physical system. The largest part
of the problem is stuff in /proc. Look in places like /proc/cmdline,
/proc/interrupts, and /proc/cpuinfo to see why.
My plan is to implement another filesystem which creates a poor-man's
overlay over the real /proc. It will have two sources of information
- the real /proc and a shadow /proc on the host. When the user inside
UML looks at a file in /proc, this new filesystem will check for the
corresponding file in the shadow hierarchy on the host and use that if
it's there. Otherwise, it will just call into the UML /proc. So, by
sticking stuff in the hierarchy on the host, the admin will be able to
override selected pieces of the UML /proc.
25 Jul 2002
2.5.28 is out. The big news is that UML again got dropped. So, I get
update and send it in again. The big change this time around is all
the irq changes. They are going to cause this UML update to be more
troublesome than usual. Global irq disabling and enabling is gone,
which overall is a good thing. I would hate to have to implement them
in UML. On an SMP UML, that would require sending an IPI around to
all the other processors, and not continuing until they had all
confirmed that they had fiddled their interrupts appropriately. That
would absolutely suck, performance-wise.
Rather than implement that, I would just never allow interrupts to be
handled by any processor other than processor 0. Then local interrupt
enabling and disabling is equivalent to global enabling and
disabling. With this change, it will be possible to distribute
interrupts around the processors of an SMP UML.
In other news, hostfs in 2.5 is broken. I fixed a few bugs, but there
are more remaining.
23 Jul 2002
-43 is out. The big news here is that is has SCSI support.
Currently, there is only the scsi_debug driver, which operates in
memory. I'm scheming to split the file I/O code from the ubd driver,
leaving behind an interface to plug it back into, and doing the same
to scsi_debug. This would allow the ubd file and COW code to be
plugged into the SCSI subsystem, and allow the in-memory device from
scsi_debug to be used as a ubd device.
Another cute thing is /proc/mconsole. It is created if UML is booted
with an mconsole notification socket. Anything written to it will be
sent out to whatever is listening to that socket as a notification.
22 Jul 2002
I sent the latest UML to Linus. We'll see how this one goes.
21 Jul 2002
Linus released 2.5.27 yesterday and it did not contain UML. Oh well.
So, I'm having another go at it. I released -42 today, which is
mostly cleanups and bugfixes to -41. There are also some driver build
changes from Henrik.
I dropped this into 2.5.27 without too much trouble. So, I'll package
it up and send it in to Linus again tomorrow.
18 Jul 2002
I released 2.4.18-41 today. This included a new way of setting up
xterms which gets UML out of the business of allocating
pseudo-terminals for them, which has been a source of trouble for a
while. As a side-effect, the terminal emulator is now configurable.
As another side-effect, it should now be possible to easily run and
control the UML debugger from a script, like the UML test suite.
I also merged a bunch of the build cleanups from 2.5. Some of my
Makefiles were seriously obsolete, and now they are slightly less so.
I made a tools release, which adds a jail kit. This contains the
tools needed to run UML as a non-privileged user inside a right chroot
jail.
17 Jul 2002
I announced 2.5.26 to the world today. Having done that, I split the
patch into two pieces, one containing changes to generic files and one
containing everything under arch/um and include/asm-um. This is the
way Linus said he liked to get ports when I asked him about it in
Ottawa.
Missing from this for now are hostfs, the tty logging patch, and the
page validation patch. I'm going to send hostfs in separately. The
tty logging patch will require advice and consent from whoever owns
the tty driver (and is willing to admit to it). And the page
validation patch will have to go in separately as well, considering
the reaction to it when I brought the topic up on LKML.
It's all off to Linus (and the generic piece was cc-d to LKML). We'll
see if this fares any better than my previous attempts.
16 Jul 2002
Linus released 2.5.26 today, so I updated. No problems with it.
There weren't any arch changes in this version.
15 Jul 2002
2.5.17 and 2.5.18 required some include tweaking because the tlb
flushing stuff moved out of pgalloc.h. Other than that, there were no
problems.
2.5.19 moved some stuff from arch code into generic code, which is
usually a good thing.
2.5.20 renamed the swap entry access macros, which was no big deal.
2.5.21 also posed no problems.
2.5.22 introduced some large build changes which broke some of my more
antiquated Makefiles. After some modernizing, they started working
again. There were also some kdev_t changes which broke hostfs.
The hotplug CPU changes in 2.5.23 required some minor changes. 2.5.24
did nothing but move sys_pause from arch code to generic code. 2.5.25
rearranged page fault handling a little, made a small build change
that had spectacular results, and changed sys_sched_yield to plain old
yield. After I got it to build, there turned out to be a division by zero
problem caused by the HZ changes. jiffies_to_clock_t is defined as
which blows up when HZ < USER_HZ. UML HZ was 52,
while USER_HZ is 100. I contemplated fixing this by turning the
nested division into a multiplication, but that would have just made
the calculation vulnerable to overflows. So, I just bumped the UML HZ
up to 100.
14 Jul 2002
On to 2.5.8. Nothing major, a couple new system calls, and some
header file rearrangement. It compiled and booted without too much
trouble.
2.5.9 was no problem at all. I fixed a glitch or two, built it, and
booted it. So were 2.5.10 through 2.5.13.
2.5.14 was more interesting. The 2.5 scheduler bug showed up here
with a vengeance while it hadn't with anything previous. The problem
is that the O(1) scheduler calls the arch switch_to with interrupts
blocked. With UML, this means that a SIGIO can arrive before SIGIO is
forwarded to the incoming process, and it will be pending in the
outgoing process until it is scheduled again. This may never happen
because that SIGIO could be a disk IO completion that is necessary for
anything to run again. So, I applied my old fix, which makes the
problem disappear. This checks for pending SIGIO after the
forwarding, and does an explicit kill(next_pid, SIGIO) if there is one
pending.
tmpfs fails to mount because the superblock allocation is somehow
failing with -ENOMEM. And the whole boot is a mess, with -EIO
appearing all over the place and /proc failing to mount for some
reason.
It turns out that the filesystem was messed up. And it turns out that
a little message that I was seeing was one I put in to make sure
I checked some code that had never run. It started running now
because of changes elsewhere in the kernel, and it turns out to be
buggy in such a way as to cause data corruption. So, with that fixed,
I get nice clean boots, except for a panic cause by the failed tmpfs
mount.
And that turns out to have been caused by an incomplete merge of the
memory stats change. Those stats are now kept in generic code rather
than in the arch. tmpfs calculates the number of available inodes
from the number of pages of available memory. I forgot to delete the
UML declaration of totalram_pages, which meant that the generic value
stayed at zero. Fixing this eliminates the mount failure and the panic.
With that all settled, I move on to 2.5.15.
2.5.15 is uneventful. The only thing of interest was a little signal
delivery bug fix which I had spotted when it was first sent to LKML
and which I already had in my pool. After the patch went in, UML just
compiled and booted.
2.5.16 made jiffies go away as a normal variable. The other arch link
scripts all create jiffies as a symbol aliased to jiffies_64. This
booted after adding that to the UML link script and removing the
CONFIG_SMP from around the definition of mmu_gathers.
13 Jul 2002
Continuing whacking away on 2.5.5. After fixing page table things
that I had messed up, it finally booted.
2.5.6 adds a new system call and changes the interface to blk_ioctl.
Fixing these up, and disabling jffs2, which was broken, resulted in a
working UML.
2.5.7 added sys_futex and applied the fs.h crapectomy to a bunch of
filesystems. nfsservctl is now apparently an optional system call, so
the system call table needed fixing for that. With those fixes, it
builds and boots.
12 Jul 2002
On to 2.5.5. This one was mostly page table changes. There were
interface changes and some new interfaces added. Someone decided it
was a good idea to get rid of the little caches for pgdirs and page
tables. I agree, since specialized little caches like that hold on to
memory that should be available to the rest of the system.
I also discovered that hostfs needed to be updated because of the fs.h
crapectomy that happened in 2.5.3. I don't know why I didn't see this
before. Basically, in order to eliminate the header file horror show
in fs.h caused by the inode union needing to have an entry for every
possible filesystem, the filesystem-specific data now includes the
inode rather than the inode containing the filesystem-specific stuff.
11 Jul 2002
I decided to get on the stick and start getting UML going with 2.5.
My 2.5 pool had 2.5.3-pre5 in it, so that's where I started. I
grabbed all the patches from there to 2.5.25 and started whacking
away.
I got 2.5.3 and 2.5.4 compiled and booting. I put off further testing
in the interest of making progress through 2.5. I am generating UML
patches along the way, and I'll exercise them more heavily later on.
The major work needed for 2.5.3 was the block driver. It needed
updating for the bio changes. Nothing major, except it confused read
and write requests, resulting in a trashed COW file when it tried
reading the superblock. I didn't realize this for a while, and was
trying to figure out why it wasn't reading the superblock even after I
fixed the read/write confusion.
2.5.4 changed how task_structs are allocated. They used to be at the
bottom of the kernel stack. Now, there is a minimal thread_info
struct there, and the main task structure is allocated by kmalloc just
like everything else. This took a fair amount of time to fix enough
so that UML would compile. Once that happened, I had to chase down a
few bugs that assumed that the current task_struct was at the bottom
of the stack.
10 Jul 2002
I started releasing rapid-fire patches. I just finished with -40.
There's been some more stuff moved under the OS interface. Also a
bunch of fixed bugs. I discovered that some gdb features that I
thought were working weren't. So, I fixed them. I also discovered
that closing a terminal at one end doesn't cause a SIGIO at the
other. So, I added that to the SIGIO emulation.
6 Jul 2002
Harald Welte is now winging his way back to Germany. He had been
staying with me for the week after KS and OLS. I fear his most
lasting impressions of lovely New England will be the insects... Oh
well.
I decided to start thinking about making UML OS-portable so that
Chandan Kudige can start merging bits of the Windows port. I created
a new directory to hide Linux specific code in and defined an
OS-independent interface for it (and other OS ports) to implement.
It's fairly rudumentary right now, containing a handful of file and
process operations, but the overall intent should be fairly clear.
So, with that, I released -37. It consists almost solely of the code
reorganization (the exception being an updated config.release).
2 Jul 2002
I'm back from the Kernel Summit and OLS. In KS news, it turns out
that Linus likes UML. Alan apparently clued him in that people
are using UML for real work, and they care about performance. He has
no problem with exposing address spaces to processes, nor with having
UML doing an address space switch in conjunction with a signal
delivery. This would essentially be sigaltstack_mm, with an mm switch
as well as a stack switch. This would have a number of advantages,
including making 'jail' mode trivial, cleaning up UML, speeding up UML
context switches, and giving UML a larger virtual address space.
In OLS news, my talk went pretty well. I described the major things
that I want in order to make UML run better, and there were no
arguments with them. They were all non-controversial, so it looks
like they're going to happen. The downside is that there is a 2.5
function freeze for Holloween this year, so it all needs doing by
then.
I departed from tradition slightly and annotated the slides before the
talk. Since this was a brand-new talk (so brand-new that I finished
it 15 minutes before the talk, which gave me just enough time to get
over to the conference from the hotel), I did the notes so I'd have
some idea what I should cover on each slide. I couldn't look at the
notes during the talk, but just doing them helped me remember what I
wanted to say for a given slide. This also has the advantage that
they can be put on the UML site immediately, rather than when I get
around to annotating them. So,
here
they are.
A few other UML tidbits:
There was lots of demand for a 2.5 UML. This, combined with the
Holloween deadline, means I need to get moving on this somewhat soon.
The UML swsusp patch apparently sort of works. This surprised me,
since the last news I had was that it didn't resume properly. So, I
probably should look at it and see about integrating it into UML.
Michael Richardson of the FreeS/WAN project did a talk on using UML as
a testbed for regression testing. It was fairly interesting and
well-received. He demoed it with four UMLs running on a FreeS/WAN
server someplace outside the conference. It went well, although the
connection was a bt slow. Richard Briggs was in the audience and was
running a six UML testbed on his laptop, which he offered as a demo
when the connection to the external demo was looking a bit iffy.
Bert Hubert almost used UML to demo something during his talk. He
ended borrowing another laptop and using it instead, since that setup
would be somewhat more authentic than using UMLs. Oh well.
21 Jun 2002
-33 turned out not to build unless CONFIG_TTY_LOG was enabled. So, I
released -34 a few hours later with the fix.
Today, I'm going to release -35 with gdb stack switching implemented.
This lets you detach from the thread that's currently in context,
attach to an out-of-context thread and look at its stack. This makes
it a lot easier to debug deadlocks since you don't have to manually
reconstruct stack traces from hex dumps of the stacks. I, of course,
have gotten used to the hex dump approach, so I didn't see what the
problem was. Other people did have problems with it, including Peter
Braam and Cluster File Systems, who offered a small bounty for this
feature. And, now that it's here, I do have to admit that it's pretty
handy.
This will be the last UML release for a couple of weeks since I'm
getting ready to head up to Ottawa for the Kernel Summit and OLS.
Those will be next week, and I've got Harald Welte visiting the week
after, which means that work might be thin for that week as well.
18 Jun 2002
Last night, I added /proc/exitcode, which allows a process to set
UML's exit status. This is useful for one-shot UMLs, which run one
thing and exit. It lets the thing running inside UML export its exit
status to the outside caller.
So this will be -33. It'll be available as soon as I get the web site
build process all put together again.
17 Jun 2002
I spent a couple fruitless days looking for whatever is causing JVMs
to crash under UML. It appears to be related to signals somehow,
since the thread that segfaults is always returning from sending
another thread a SIGRTMIN. It first appeared that the other thread
was provoking the crash, since there was always a context switch from
the sending thread to the receiving thread before the sending thread
came back into context and returned from the kill. However, I
suppressed context switching in that case, and it still died on return
from the kill.
The code that's segfaulting appears to be generated by the JVM rather
than being the JVM itself. It is one of very many small pieces of
code pointed to by a large table. Each of these is about 6
instructions long, starting by dereferencing a pointer, doing one or
two instructions worth of work, calculating an index, and jumping to
whatever code is pointed to by that entry in the table. The crash is
coming at the very start of one of these blocks. The register that
it's dereferencing contains zero, which is bad.
I decided to put that off for the moment and do some other things.
So, I redid the web site build. My major complaint was the speed at
which it rebuilt. The big culprit was all of the changelogs that have
accumulated. They are all generated from a large and growing XML
file, with each entry getting its own web page, and requiring the
processing of the entire file to generate. Needless to say, this is
an O(n^2) operation, and I was well within the quadratic regime.
So, I redid it, adding some infrastructure that allows rebuilding of
files only when they've really changed. So, the fact that
changelog.xml changed does not cause the rebuilding of all of the
changelog-*.html files, even though their source file changed.
I also tidied up the dependencies and reorganized the pool itself to
make it a bit cleaner.
In actual development news, I integrated the honeypot tty logging
patch into the main pool, along with a little patch I got from
geoffrey hing. I also added the ability to log to a preconfigured
file descriptor. This is for the benefit of chrooted UMLs, so they
can log to a file outside the jail.
11 Jun 2002
Hmmm, long time no diary entries. Well, I've been busy. I've started
getting regular offers of contracts to do UML-related things and I've
been accepting some of them. This slows down the pace of UML
development, unfortunately, but it does tend to fatten up the old bank
account, which has to be considered a good thing.
Recent development has featured a couple common threads. One is
moving UML into its own memory. Before, UML would allocate a physical
memory region which did not include its own binary. This effectively
meant that UML text, static data, and heap were not in physical
memory. This caused problems for the swsusp effort on UML because it
wants to copy physical memory out to disk. Fixing that took several
patches to get right, but it seems to be fine now.
Another theme has been openpty. This started with some mysterious
segfaults seen by a few people which ultimately were tracked down to
calls to openpty that UML was making. openpty has a larger than usual
stack frame, so when it runs on a kernel stack, it overflows it and
corrupts whatever lies just below.
My first attempt to fix it put it in a separate thread so that the
pages it used would be COWed and it wouldn't change any UML memory.
This was stupid because UML memory is mapped MAP_SHARED exactly to
defeat COWing. Attempt number two involved having the tracing thread
run openpty. This was good because the tracing thread has a proper
stack. Unfortunately, openpty can call malloc, which gets converted
into kmalloc, which is a very bad thing to call from the tracing
thread. So, my current code goes back to the original mechanism,
except that a larger than usual stack is allocated in this case. This
should be OK now.
With these things settling down, I'm about ready to release -32, and
I'm thinking that a full release would be good to do soon, as well.
Maybe later this week or over the weekend.
17 May 2002
A week or so ago, Steve Freitas kindly assembled and sent me a nice
little SMP box so I could chase the host SMP bug that UML is
exercising. I got around to fitting it into my environment - it
caused my network to outgrow my crossover cable, so I invested in a
switch and a bunch of cables. I've got it on the net with a serial
line console running to my desktop box, which is all running fine.
UML also runs fine on it, which is disappointing. It'll be hard to
find the host bug if I can't reproduce it.
In development news, I did a bunch of work on ptrace, which forms the
bulk of -26, which is released today. Watchpoints now work in gdb
inside UML. Kernel watchpoints don't work yet, but they will soon. I
fixed a couple other ptrace bugs, including one which could be used to
break out of UML.
11 May 2002
The big UML news is that it is now self-hosting. This is more a
demonstration of UML maturity than something that's very useful. It's
still nice to be able to do, though. There's a description of how to
do it here.
UML development has been concentrating on fixing bugs, as usual. The
last few releases have mostly consisted of small fixes.
The test suite received an overhaul. It is now willing to build UMLs
according to the needs of the tests. So, tests can now specify how
they want UML to be configured, and the suite will build a UML if
necessary.
28 Apr 2002
I'm back to knocking items off my todo lists. At this point, I'm down
to about 75. The recent victims are mostly small items that had
accumulated with some other bugs that people confirmed had already
been fixed.
I released -21 today after getting rid of most uses of tracing_cb,
which UML threads use to request the tracing thread to create
processes for them. Having UML threads do it themselves exposes that
code to the UML gdb, as well as get one step closer to having
miscommunication with a helper hang only the thread that started it
rather than the whole UML. To get there also requires that input from
helpers be handled asynchronously rather than synchronously as is the
case now.
25 Apr 2002
This was a fairly lazy week. I went down to West Virginia at the
invitation of David Krovich to give a talk at WVU. That went fairly
well - the chairman of the CSEE department was apparently impressed by
the number of students who attended. After dinner, I gave another
talk at a MORLUG meeting (MORLUG == Morgantown LUG) which consisted of
me firing up UMLs and demonstrating various neat things you can do
with it. This also seemed to go well.
In extra-curricular activities, we tried to go hiking on Sunday, but
got rained on heavily. It looks like nice country if only the clouds
would get out of the way so you could see something.
In UML news, I spent some time fixing the iomem support. It was
broken in such a way that I had a hard time believing it ever worked.
What I now think is more likely is that the VM system changed in a way
that broken iomem, but no one noticed. The problem was that the VM
system deals largely with page structs instead of raw page addresses
and the iomem regions had no sane mem_maps, so they had no page
structs.
I fixed this by changing the infrastructure to allow for segmented
physical memory consisting of regions which have their own separate
mem_maps. This will allow for plugging and unplugging of iomem
regions. I was hoping this would work for physical memory regions as
well (plugging anyway; unplugging is harder), and after a bunch of
failed experiments, decided that this wasn't going to be.
After getting this working, I decided to release 2.4.18-19. It also
contains James McMechan's partitioned device support and a bunch of
smaller bug fixes that were noticed by various people.
15 Apr 2002
More bug-bashing. My lists contain a total of 93 items, but a large
number of them are about to die. The umlgdb expect script from
Chandan Kudige will knock off the two related to reloading module
symbols. When I verify that UML can boot as a diskless client, that
will get rid of five more. James McMechan's ubd partition patch is
currently accounting for six items. I should be able to knock all
those off in the next couple of days.
I released an RPM last Tuesday to get the accumulated changes out to a
wider audience. I think I'm going to start releasing an RPM every two
or three weeks from now on. 2.4.x releases are too far apart for me now.
6 Apr 2002
I spent the last few days on a bug-smashing spree. To-do items had
accumulated at an alarming pace over the last couple weeks, so I
decided to whack away at them a bit. I had over 110 items on my
various lists, so I knocked off as many as were easy to kill. I now have
99 items, so I got rid of more than 10 of them.
Prominent among them are
floating point registers not being available to gdb inside UML or
stored in core files
hostfs not being able to create unix sockets
the daemon transport now gets its MAC from uml_switch
if the umid is set on the command line, it is put into host process
names and into xterm title bars
There were some small patches from mulix, Sapan, and Daniel Phillips,
which all did useful things.
With that, I released 2.4.18-14 and uml_utilities_20020406. This may
be the basis of another full release (with an RPM instead of just a
patch). I need to do that soon since the 2.4.18 UML is getting fairly
old at this point.
3 Apr 2002
In the nearly two weeks since the last entry, I've pretty much knocked
off the console flow control bug (the one remaining piece is to make
sure that it works correctly when ptys deliver output SIGIO) and the
init hang on older machines. These are two of the oldest UML bugs.
The init hang is caused by init executing a cmov on a processor that
doesn't support it. What I ended up doing (at the suggestion of Alan
Cox who saw this on one of his boxes) is detect cmov support by
looking at /proc/cpuinfo, then panic if init gets a SIGILL on a cmov.
In other news, I got all of Bill Stearns' bootable filesystems over to
SF, so the links to them are likely to work. They're not on
ftp.nl.linux.org yet though.
22 Mar 2002
I've decided that I'm out of major design changes I need to make and
can now concentrate on knocking items off the todo list. The one
published on the site has 44 things on it. I also keep a todo mail
folder containing pieces of mail that describe something well enough
that I can keep track of it. That folder contains 54 messages right
now, so I've got almost 100 things to do. Some are duplicates, and
some are already fixed and I haven't figured it yet, so the actual
number is somewhat smaller.
My latest run of the test suite succeeded in booting all of the
filesystems available from the UML site, which is probably the first
time that has ever been true.
15 Mar 2002
I discovered a remnant bug. 'strace -p' didn't work. Fixing that was
easy enough, but I decided to clean up some code while I was in it.
That took more work than I expected, since it involved saving state in
the thread structure in signal handlers, and when you forget to
restore the old state, you get very obscure misbehaviors. I ended up
backing out the changes and putting them back in one line at a time
before it dawned on me that perhaps restoring the old stuff would be a
good plan.
So, I released 2.4.18-7 today. I also released another version of the
test suite. This one allows tests to be interactive and for the perl
test driver to interact with them.
In separate news, my paper proposal for OLS was accepted. So, it
looks like I'm on the hook for another paper.
14 Mar 2002
After much surgery, I finished the pt_regs to sigcontext work and put
UML back together. There were surprisingly few bugs to chase once it
compiled again. I fixed around three bugs, and haven't found any more
since.
I did some more work on the test suite. There is now a kernel build
test, for which I wrote a perl mconsole client. This prompted me to
add the option of the mconsole driver sending the name of its socket
to a socket specified on the command line. This allows scripts to
find out where to send mconsole commands without having to parse the
boot log. It also tells them when UML has booted to that point.
This test also exposed a bug in the mconsole driver which caused
commands not to be NULL-terminated.
7 Mar 2002
Bad news. I decided to do a UML-sucks/rocks-ometer on Google. Well,
"sucks" beats
"rocks" 125-83. To
make things worse, the first few "sucks" entries are me.
4 Mar 2002
I made the 2.4.18 announcement on freshmeat today. That is the final
piece of this release.
I have known for a long time that UML is susceptible to bus errors in
random places in the code if it touches memory that it had previously
mmapped, but the host can't back with physical memory. This turned
out to be a lot easier to trigger than I expected. I run with tmpfs
mounted on /tmp for speed reasons. It has a maximum size of 1/2 RAM
by default, which is 128M for me. I got a UML to hit that limit and
crash with a bus error.
So, I posted an RFC to lkml asking for comments on my proposed
solution which was to add a hook to __alloc_pages to allow the
architecture to touch pages before they're returned to the caller.
Physical architectures have no need of this because they have a known
amount of physical memory and they know it's not going anywhere. UML
doesn't, so it needs to assure itself that an allocated page is real
and that accesses to it won't fault.
What I got was an argument with Alan Cox who persistently doesn't
understand what I'm talking about. He has appeared to believe that
I'm trying to get good behavior when the system is out of memory and
good behavior is impossible, that I want the host kernel to allocate
memory as soon as the address space is allocated, and a bunch of other
things that I don't begin to fathom. Peter Anvin hopped in briefly
with what look like similar problems.
So, I implemented what I wanted, and sent the patch in. Hopefully
that will clear out the confusion.
In actual development news, I'm converting UML from storing register
information in pt_regs structs to storing it in sigcontext structs.
What I'm doing is making UML return to userspace by way of host signal
returns rather then having the tracing thread teleport it back with
PTRACE_SETREGS. The main advantage is that it is guaranteed to
correctly restore floating point state, which has worried me for quite
a while. It has other advantages, like removing code from the tracing
thread, which will make it easier to finally eliminate at some point.
1 Mar 2002
I finished the release of 2.4.18 today with the bare kernel and the
RPM. No deb since Matt is taking care of that for me now.
I cleaned up the test driver some more. The local configuration
information is now stored in ~/.umltest, and it is possible to pass in
configuration options on the command line. This is in preparation for
automatically running the tests whenever a new patch is ready.
25 Feb 2002
I released the 2.4.18 UML patch today. It was a no-effort update,
except that there were a bunch of placeholder entries for
the extended attribute system calls.
I'm going to sit on the full release for a few days while I work up a
new test suite and harness. Hopefully, I can get something automated
set up so that whenever I release a patch, it will get banged on
without me having to do it by hand.
24 Feb 2002
I think I'm emerging from signal delivery hell. Everything seems to
work again with the exception of a memory corruption problem which may
not be new. I've discovered new and interesting ways to screw up.
It turns out that when you leave -ERESTARTSYS or -ERESTARTNOHAND in %eax
when leaving a system call, the host kernel will helpfully subtract 2
from eip. This is because a system call instruction is 2 bytes long
and it restarts them by executing that instruction over again. This
turns out to be a problem when UML itself is restarting one of its own
system calls. It delivers the signal first, so eip points to the
beginning of the handler, and if you don't put a zero or something in
eip, then you will fake the host into subtracting two from eip. That
puts it at the very end of the previous procedure, which will try to
return from a stack frame that never really existed in the first
place. This leads to very interesting debugging sessions.
It also turns out that ptrace "knows" that when it has intercepted the
start of a system call, eax contains -ENOSYS. It depends on this.
When I changed how UML gets the process state before a system call, I
changed the value in eax to something else, and ptrace stopped
working. It took a couple of days to figure this out.
But things seem back to normal now. My signal delivery exercisers
work OK, my stress tests work OK, UML works when compiled on 2.4 and
run on 2.2, and vice versa, which -13 didn't. So, I'm releasing -14
now and maybe this signal delivery rewrite will be done.
17 Feb 2002
I submitted a paper proposal for OLS yesterday. This time, I decided
to rant and rave about how the host kernel needs to be fixed to better
support virtual machines. We'll see how that goes. At least it's
different from the standard UML song and dance I've been flogging for
the last couple of years.
I redid the UML signal delivery code in order to fix the pthreads hang
with the newer pthreads library. I prototyped this in a small
standalone process in Incheon Airport (Seoul's international airport)
on my way back from Brisbane and just got around to integrating it
into UML. It massively cleaned up a bunch of code and opens the way
for getting rid of some other problems that have existed for a while.
11 Feb 2002
My flight leaves tonight, so I spent the day wandering around central
Brisbane. Ben LaHaise happened to take the same CityCat ferry (which
is a system of catamarans used to ferry people up and down the
Brisbane River) as me down to the city. He tried to blow up the boat
with his umbrella, but, fortunately, an alert crew member stopped
him.
Back to the Uni at the end of the afternoon, collect my bags, call a
taxi to the airport, and wait for the flight to Seoul. Then it's
another 14 hours to JFK and another couple back to CT and then I will
be hopelessly confused about what time it is for a couple of days.
10 Feb 2002
LCA is over as of yesterday, and I'm heading home tomorrow.
The slides from my talk (with notes) are available
here.
It was pretty well attended and it seemed to be received fairly well.
LCA always seems to do some innovative things, one of which is to
rerun talks that a large number of people regretted missing. The
three talks that were chosen this year were a virtual reality talk
that a huge number of people wanted to see, Andrew van der Stock's
talk on code auditing, and one other that I can't remember. This was
good because I was one of the huge number of people that wanted to
see the virtual reality talk. However, it turned out that my talk was
the number four vote-getter, and this turned out to be relevant when
Andrew was nowhere to be seen when the reruns were about to happen.
So, I was happily debugging the ppc UML build with Anton Blanchard
when one of the organizers ran up and asked me if I could do mine
again. I did so, except I skipped over some of the heavier parts of
the talk to leave a good bit of time for a demo at the end.
This went well, except for the panics I got when I tried to have two
UMLs mount the same filesystem. I demoed three UMLs running (one
Debian, two Slackwares) with most of them (plus the host, I think)
displaying on the X server of one of the Slackware UMLs. I showed
various other aspects of UML like what it looks like from the host
side.
1 Feb 2002
I'm off to Australia tomorrow for
LCA 2002.
This caps a fairly productive week of UML bug hunting:
mistral and blinky started seeing a panic in fork. mistral figured
out how to reproduce it and tracked it down to the point where it had
something to do with kernel threads reference counting their mm's.
This was enough information for me to fix the bug, by giving kernel
threads NULL mm's.
While I was at ISTS giving a talk, I spent some free time looking at
the pthreads problem that people have been seeing for a while. It
turns out to be a problem with UML signal delivery. I had assumed
that the registers going into a signal handler didn't matter (except
for the IP and SP, of course) and that only the stack frame mattered.
So, the process registers at the start of a signal delivery are
initialized with a set that was captured from a UML thread at boot
time. This works fine usually, except that recent pthreads libraries
store some thread-private data in %gs, and the %gs value has to be
preserved in signal handlers. So, the UML signal delivery mechanism
needs to be reworked again.
The UML IO hangs that were reported this week were tracked down and
found to be a bug in the host's handling of SIGIO. It turns out to be
possible, on an SMP host, for SIGIO to be queued to a process after
that process has returned from the fcntl that registered a different
process as the SIGIO recipient. This breaks UML badly because SIGIOs
end up queued, but not delivered, to a process which is out of context
and sleeping.
30 Jan 2002
I gave a talk at ISTS yesterday on the UML security work that I did
last week. It's available both as
html-ized
slides and as the
original Star
Office presentation. These are intended to provide a starting
point for anyone wanting to probe this for exploitable holes as well
as anyone who's curious about what was done.
I released 2.4.17-10 today. It contains a pile of bug fixes and a
bunch of changes which allow a UML patch to come close to compiling in
both 2.4 and 2.5 pools. I reverted a change which is causing problems
on SMP hosts. I decided that using sockets rather than pts devices to
communicate between the IO thread and UML was a good idea because
sockets are lighter weight and they're pretty much guaranteed to be
supported on the host, whereas there are lots of systems without pts
devices. However, there is some difference in how SIGIO is delivered
which causes UML to lose interrupts once in a while. The effect is
that it seems to hang on boot, but can be made to continue by banging
on the keyboard.
MTD is in the configuration now. So, UML supports MTD devices and
creating JFFS2 filesystems and mounting them seems to work, although
there are some nasty-looking error messages along the way. They don't
seem obviously related to UML though.
25 Jan 2002
The security work is largely done. The exception is the lcall
prevention fix that's needed on the host. So, I released 2.4.17-9
today. It also contains a number of fixes and patches from other
people, the largest being the latest set of James McMechan's ubd
changes.
23 Jan 2002
I released 2.4.17-8 yesterday (and 2.4.17-7 earlier this week without
dignifying it with a diary entry). I spent a fair amount of time
tracking down some old debugging problems. The strace recipe on the
debugging page
hasn't worked for a while, so I mostly fixed it. strace still doesn't
see system calls from new processes until they receive a signal.
I also figured out what was happening with using the gdb under ddd as an
external debugger. ddd periodically calls wait on gdb, and when UML
attaches it, gdb gets reparented away from ddd. wait starts returning
ECHILD, and ddd reacts by shutting down gdb's input, and gdb, in turn,
exits. To fix this, I think I'm going to have to have a 'gdb-parent='
switch that will make UML attach to the parent and fake normal return
values from wait.
In other news, there have been a couple of articles about UML
recently. NewsForge ran
one
yesterday. This is a followup on the article last week about Linux
virtual machines which completely failed to mention UML. Bill Stearns
also noticed
this article on using UML as the basis of a honeynet.
I've started finishing off the security work needed to make UML a
secure root jail. The 'jail' switch now checks for config options
which would make the UML inherently insecure and refuses to run if any
of them are enabled. Currently, the proscribed options are
CONFIG_MODULES, CONFIG_HOSTFS, and CONFIG_SMP. CONFIG_MODULES is
fairly obvious. If modules are enabled, then root can insert any code
at all into UML, and a nasty root would insert code that execs a shell
or something on the host. CONFIG_HOSTFS is forbidden to prevent
accidentally providing access to the host filesystem. This is a bit
dubious because hostfs is not inherently insecure, and I may relax
this one at some point. CONFIG_SMP is non-obvious. Lennert Buytenhek
noticed the relationship between SMP and security. 'jail' is
implemented by unprotecting kernel memory (by making it writable) on
entering the UML kernel, and write-protecting it on kernel exit. If a
process were to have two threads, one busy-waiting in userspace, and
the other sleeping in the kernel, kernel memory would be writable
because the sleeping thread would be in the kernel. So, the spinning
thread would wait for that to happen and write on whatever part of
kernel memory would let it escape. This will be fixed when UML gets a
separate address space for the kernel.
I've stared at /proc and /dev to find devices that provide access to
kernel memory. The only two that I spotted were /dev/mem and
/dev/kmem. These are disabled with a trick that someone on
#kernelnewbies told me about. Access to them is controlled by
CAP_SYS_RAWIO, so removing that capability from the bounding
capability set makes it impossible for any process to ever get it.
So, no process can ever open those devices. Even better, in my
limited testing, nothing seems to break badly as a result.
/proc/kcore looks suspicious, but it's a readonly file that seems to
fit out a memory image wrapped in an ELF header, so it's OK,
security-wise.
18 Jan 2002
2.4.17-6 is out. There are a lot of driver cleanups and bug fixes in
this patch. The IRQ hang is fixed. The default console and serial
line channel initialization strings are now configurable.
In other news, NewsForge ran an
article
about Linux virtual machines which covered everything
relevant, including some things that weren't virtual machines, except
for UML. They got some complaints about that, and later that day, I
got a piece of email from the guy who wrote the article wanting to
write a followup about UML. So, it looks like UML will be getting
another nice bit of publicity.
13 Jan 2002
I made another patch against the 2.5.2-pre tree again today. This one
is against 2.5.2-pre11. It has been sent off to Linus so he can drop
it in his bit bucket.
In another development, it turns out that the O(1) scheduler breaks
UML by holding IRQs disabled across context switches. This results in
SIGIO (i.e. from disk IO completions) to be trapped in a process that
has gone out of context, and can't be woken up until something else
notices that the IO has completed, and of course it won't because the
SIGIO has been delivered to the wrong process.
11 Jan 2002
After spending five days tracking down a swap corruption bug, I
discovered that Rodrigo de Castro had explained to me exactly what it
was about a month ago. Unfortunately, at the time, I didn't know
enough about the swap code to decide whether he was making sense. Of
course he was, and I discovered that at the end of the great bug hunt.
So, with that fix, Lennert's latest SMP changes, and a bunch of
smaller stuff, I'm releasing 2.4.17-5 today.
Linus saw fit to silently drop UML into the bit bucket again, so I'll
make another patch soon and send it in.
5 Jan 2002
I released the 2.4.17-3 and 2.4.17-4 patches this week. The biggest
change has been the merging of Lennert's SMP fixes.
I made another attempt to get UML into the Linus tree. This patch is
against 2.5.2-pre9 and is the 2.4.17-4 patch. We'll see how well this
attempt fares.
30 Dec 2001
I announced the full 2.4.17 release and the 2.4.17-2 patch today. The
patch is largely changes to allow UML to calculate current from the
stack. This is to make life easier for Lennert, who's trying to get
SMP working. It also contains a bunch of fixes for bugs that crop up
when host devices get closed from under UML consoles or serial lines.
Iain Young is making a decent stab at a UML/sparc64 port. He's got
the boilerplate filled in (albeit with some skeptical comments about
their correctness...) and he's trying to get the whole thing to compile.
Linus released 2.5.2-pre4 today with no sign of UML in the changelog.
I grabbed the patch to make sure it wasn't there. It wasn't. Grrr.
I'll give him another patch and then I'll spam him with it again.
28 Dec 2001
OK, so the diary has taken a bit of a holiday break. I released the
2.4.17 UML patch yesterday. The full release will be forthcoming. I
also ported that patch into 2.5.1, created the 2.5.1 patch, and sent
it to Linus. Hopefully, he will put it in without my having to resend
it too many times. I sent a little note off to LKML announcing this,
which got some favorable reaction, both on and off the list. Alan,
being his usual taciturn self sent a reply, which read, in its
entirety, "Cool".
This release had a lot of accumulated stuff in it. The biggest item
are the port channel, which let you attach any number of UML consoles
and serial lines to a host port, at which point you can access them by
telnetting to that port. I also redid the context switching mechanism
after thinking of a much simpler way of doing it. This should also be
much faster since it doesn't involve signals flying around, so context
switches are now invisible to the tracing thread.
8 Dec 2001
I released 2.4.15-3 today. It contains some fixes to previous patches
and a lot of changes to the gdb support. gdb now sees ^C immediately,
rather than an arbitrary amount of time after it's typed. I also
cleaned up that code quite a bit. This knocks a couple of items off
the todo list. It also
sets me up to fix the gdb shell hang, but I'll let these changes gel
for a bit before dealing with that.
7 Dec 2001
I got the Sysadmin Disaster of
the Month contest going about a week
later than I should have. The thing that happened a week earlier was
the publication of an
article that I wrote for O'Reilly on using UML to simulate and
recover from disasters.
This month's disaster is a trashed root superblock. It involves
booting UML, zeroing out the superblock, and figuring out how to fix
the filesystem. I had some trouble coming up with a good example to
use for the contest. So, if I have similar troubles at the end of the
month, I might just trash a filesystem, make it available for
download, and the contest will be to figure out what's wrong with it
and fix it.
4 Dec 2001
Back from Linux-Kongress. Back on US/Eastern. I think I never left
it, which made life (the staying awake part of it) difficult in
Holland.
As for highlights of the conference, we have:
I and the UML project seem to have some name recognition. Everyone I
talked to seemed to have heard of both me and UML, which is very cool.
The talk went reasonably well. I gave the same talk as I did at ALS.
I had forgotten that it was somewhat tailored for ALS (i.e. I tried
to avoid talking about stuff that I had talked about at the previous
ALS), and it would have been somwhat different if I had prepared a new
talk for Linux-Kongress.
I met a pile of cool people, like
Lennert Buytenhek (who, in recognition of his contributions to both
projects, was embarassed by both me and Rusty by being asked to stand
up for some audience appreciation, which must be a new Linux-Kongress
record)
Roman Zippel (who told me about a couple of ubd driver bugs, one of
which I knew of (a subtle rounding error), and one of which I didn't
(Greg Lonnon and I went to some trouble to put the COW header in
network byte order, but forgot to do the same for the block bitmap
(grrrr))
Bruce Walker (who's in charge of the Compaq SSI project, and who asked
me clustering questions during my talk, at which point, I (correctly)
guessed who he worked for and why he was asking)
Fabio Olive Leite (a Conectivite who gave me a nice Conectiva
filesystem image a while back, and whose name I unaccented to get it
through my XSL processor)
Philipp Reisner (who threatened an Alpha UML port a while back, but
gave up, so he's less cool than the others :-)
The organizers took the speakers back to Amsterdam after the
conference to spend the day bumming around the city. We broke up into
small groups and went our separate ways. Our group spent much of our
time in two smallish cafe-type places just talking about random stuff.
The train trip from Schiphol airport (Amsterdam) to Enschede (near the
German border) was interesting for a number of people. It was exactly
as described for me (2+ hours, direct train, no problem), but some
track maintenance in Amsterdam's central station caused subsequent
trains to be cancelled, so other people had nightmares involving 5
trains and a bus totalling 5 hours.
26 Nov 2001
I released 2.4.14-6 today in the interest of clearing the decks for
the 2.4.15 release. Before leaving for Thanksgiving, I had redone the
mconsole protocol to be packet-oriented. This allows a lot more
flexibility in what can be done with the protocol. As a result,
you'll need the new mconsole client for this release.
While down in CT, I redid the host channel support. It is all much
cleaner now, and makes it a lot simpler to knock a bunch of
related items off the todo list.
I decided to knock off the 2.4.15 patch today as well. It went cleanly,
aside from a ptrace cleanup and a new way of generating /proc/cpuinfo
which I had to support. I also put in the file corruption fix after
forgetting to and discovering that a boot/halt caused fsck to complain
a little.
18 Nov 2001
While I'm waiting for the meteors to arrive, I'm chasing and stomping
UML bugs. I cleaned up and released the proxy arp fixes that I did on
planes and in airports on my way to Oakland. Before, uml_net would
blindly add an arp entry to eth0 and nothing else. This is wrong if
there is no eth0, and it's also wrong if eth0 doesn't connect to the
local net or if there are other interfaces also attached to the local
net. uml_net now looks at the routing table and puts an arp entry on
every interface that talks to the local net.
I also noticed that slip support wasn't up to date, so I modernized it
and cleaned up the code while I was at it. You can now change the IP
address of a slip-based interface and the host configuration will be
updated just like the other transports.
I added some RT signal support. SA_SIGINFO is now supported, which
will hopefully fix some of the strange process behaviors that have
cropped up lately. If this fix doesn't do it, I chased down another
bug which was causing rt_sigsuspend and sigsuspend to return incorrect
values. This was causing the libc sigsuspend to hang, and its process
with it. This fixes the pthread_create hang that Greg Lonnon noticed,
plus the gdb hang, I think. I haven't checked that yet.
Those fixes are in 2.4.14-3um which I just released. You'll need the
latest utilities in order to use the network, since I bumped the
uml_net version again.
14 Nov 2001
OK, I'm back from ALS. My talk was on the first day, and it was
reasonably well attended considering the somewhat dismal overall
number of attendees. It was a half-hour talk, so about an hour
beforehand, I took my OLS slides, threw out more than half of them,
and updated the rest. That worked out reasonably well, but 30 minutes
makes for a very short talk without much detail.
Daniel Phillip's talk was the last of the conference, and was somewhat
interesting. He had a pile of raw data that he needed to turn into
slides, and all of the KDE presentation tools blew up on him in one
way or another. So, in the break before his slot, he grabbed Stephen
Tweedie, and they plus me and another guy went off to a local dim-sum
place. Daniel and Stephen sweated over Star Office on Stephen's
laptop making slides. In the event, they turned out rather well.
I left just after Daniel's talk, so I missed out on some of the
socializing that afternoon and evening. It turned out that Daniel and
Larry McVoy were talking about his clustering ideas (MetaLinux, or
ML), and it occurred to them that UML was not only a good simulation
tool for ML, but that it actually implements a good part of what Larry
has in mind. I found out about this later, and had a long talk with
Larry on Monday, in which he explained his plans. I had heard various
mumblings about it, and saw a
slide show
that Larry has, and remained unenlightened. It turns out, that as far
as I can tell, the only way to find out what he's thinking is to have
him explain it in person. Anyway, I became enlightened after our
chat, and it looks like this could be a whole new area that UML could
branch into.
In actual development news, I fully released 2.4.14 today. En route
to Oakland, I fixed uml_net so it's smarter about doing proxy arp. It
figures out what devices are connected to the local net and only sets
proxy arp on those. As a side-effect, if the host is totally
isolated, then you don't get scary-looking error messages when it
tries to set proxy arp on eth0 and it turns out not to exist.
This happened to me at OLS when I tried to demo it after my talk. I got
this nasty message which convinced me that the network all of a sudden
didn't work, and I was all apologetic and had no idea what happened.
In reality, the network was fine, and I could have demoed it if I had
retained a bit more presence of mind.
This isn't in the 2.4.14 release because I'm not happy about the
cleanliness of the change. I'll probably clean it up for the next
2.4.14 patch.
On a much-delayed train from San Francisco to Mountain View (a
supposedly 1:13 hour trip that in reality required about 1:50 and two
trains), I also figured out why you can't talk to eth1 from the host
if you configure both an eth0 and eth1. It turned out to be the same
bug that other people had noticed causing dropped packets. I was
checking errno incorrectly. I had code that did this:
n = read(...);
if(errno == EAGAIN) return(0);
forgetting that successful system calls don't necessarily set errno to
zero. So, the eth1 read was succeeding, but errno was still EAGAIN
from the eth0 read.
In other news, beware of kernels built with gcc 3.0.2. I got a
complaint from Jens Axboe today about UML leaving all kinds of
not-quite-zombie processes lying around. I looked at it a bit and
guessed that the host kernel was messed up somehow. He looked at
that, decided I was right, and that the culprit was the latest gcc.
The interesting thing was that, until he ran UML on that kernel, it
looked just fine to him.
6 Nov 2001
In preparation for fixing the problem of the console driver losing
output, I ported the SIGIO handler to use poll instead of select.
This was mostly what 2.4.13-4 was. I later discovered a bug in it,
which is fixed in -5.
I then decided to fix the problem of UML not being able to be
interrupted and backgrounded. The problem was that all UML processes
are in the same process group, with all of them stopped except for the
one that's actually running. The problem is that when UML is
backgrounded, the shell sends a SIGCONT to the process group, which
wakes up every UML process, which is very bad.
I did some failed experiments with setpgrp/setsid and friends, and
discovered that a separate process group wouldn't work because then
those threads can't write to the terminal because they're in the wrong
process group.
So, I decided that out-of-context processes should be asleep rather
than stopped. This required redoing the task switching code. They
were stopped because the tracing thread intercepted a signal from them
when they went out of context and never continued them. Having them
sleep would require that the tracing thread stop doing that and that
the threads involved in a context switch arrange the transfer
themselves.
So, what is now done is that non-running processes are asleep in
sigsuspend, and they are woken up by the going-out-of-context process
sending a SIGTERM. Races are avoided by having the SIGTERM sent
inside a section of code that has blocked SIGTERM. SIGTERM is
re-enabled atomically with the sleep with sigsuspend.
So, that plus the poll fix is the contents of -5.
3 Nov 2001
Time to patch-bomb Alan again. I sent in ten patches to get the ac
tree current with CVS. Here they are:
Miscellaneous fixes - some build cleanup,
a config update, some name changes in the mconsole driver, exporting
of gprof symbols, and various other small cleanups and fixes
A signal handling fix which eliminates
most of the process segfaults which people had been seeing
Jorgen Cederlof's context switch speedup ,
which also includes some VM fixes that I found
A sychronization patch which
includes a grab-bag of changes which I hadn't managed to get into the
ac tree yet
2 Nov 2001
That last patch went into -ac6, so the ac UML builds and works again.
The next job is to get the ac tree up to date.
I released a new utilities tarball today. uml_net should now do proxy
arp correctly. uml_mconsole is now able to take a command on its
command line and execute it, rather than being strictly a command line
tool.
30 Oct 2001
I decided to make the -ac UML build again, so I made
this patch and sent it off to
Alan. The rest of the updates will be forthcoming.
29 Oct 2001
Today is 2.4.14-3 day. I decided to remove the code in fix_range
which unmaps pages whose ptes say they're not present. That basically
caused it to try to uselessly unmap all of its unused address space.
So, I did that and it uncovered a bug. It turns out that swapped-out
pages weren't marked as needing to be remapped. Everything worked a
lot better with that fixed, and context switching should be a bit
faster now.
28 Oct 2001
I released 2.4.14-2 today. This contains the fix for the process
segfaults and the gdb problems people have been having. It also turns
on morlock's context switch optimization which I disabled until I
figured out the segfaults.
26 Oct 2001
I finished releasing 2.4.13 today.
After some prodding from Greg Lonnon, and after he did some investigation,
I figured out what the problem with gdb inside UML is. The signal
handlers don't save their registers in the thread struct. This means
that when a SIGTRAP from a breakpoint comes in and it gets forwarded
to gdb, when it gets the registers to find out what the ip is, it gets
an old, bogus value. So, it doesn't recognize that as a breakpoint
and complains about a spurious SIGTRAP instead.
25 Oct 2001
I spent the last few days chasing a process segfault problem. I
finally tracked it down today. It turns out that my rewrite of the
process signal delivery code was broken in the case of a signal being
delivered from an interrupt handler rather than a system call. It
grabs the process registers from the thread structure, saves them away
on the stack, and then restores them to the process when the handler
finishes.
However, interrupts don't save their registers in the thread
structure, so those registers represent the last system call, which
has already finished. And restoring those causes great confusion in
the process.
20 Oct 2001
I released 2.4.12-3um and 2.4.12-4um over the last week. -3 fixed a
couple of problems with -2, and -4 adds some miscellaneous fixes to
that. The major ones are that physical memory protection is optional
(controlled by the 'jail' switch) and that the network driver backends
now collect uml_net commands and output and nicely printk them instead
of having the output just dumped to the terminal. To support this,
uml_net now hangs on to the commands it runs and the output they
produce and send them back to UML. This required that the uml_net
interface be incremented, so it's now at 3. The new drivers require
the new uml_net, so if you grab the UML patch, also get the latest
utilities tarball too.
13 Oct 2001
I released 2.4.12-2um today. It's almost entirely changes sent in by
other people, dominated by Adam Heath's cleanups. There were also some
ppc fixes from Chris Emerson, and small fixes from other people.
I also released a new utilities tarball. The one change was to
uml_net, which does proxy arp in a different, and apparently more
robust way than it used to.
11 Oct 2001
Linus released 2.4.11 and 2.4.12 two days apart. I had the 2.4.11
patch uploaded, and had started releasing packages when 2.4.12 came
out. So, 2.4.12 is out there and I'm doing the packages again.
8 Oct 2001
I should have mentioned the latest -ac patches already since they've
been in Alan's tree for a few days, but I didn't, so here they are
In other news, with the help of Paul, I tracked down an ancient
console driver bug that held on to a struct tty after it had been
freed and subsequently caused panics.
I released 2.4.10-7um today with that fix and some other minor changes.
5 Oct 2001
Paul Larson found a test case for the signal problems that was
reproducable for me. So, with that in hand, I tracked down the bug,
and released 2.4.10-6um.
The bug turned out to be a result of moving where state is saved
before a signal is delivered to a process. The process registers and
some other things need to be saved on the process stack so they can be
restored later. The way it used to work is that
handle_signal would figure out what the interrupted system call
eventually returns
that value is passed up the stack and stored in the process registers
stored in its task structure
the process would be sent a signal so it starts running on its process stack
the UML signal handler copies the register state from the task
structure to its own stack
it calls the process signal handler
and restores the registers back to the task struct
What I implemented did this
handle_signal figures out what the interrupted system call eventually
returns and constructs the process stack frame, copying the registers
from the task struct onto the stack
that value is passed up the stack and stored in the process registers
stored in its task structure
The bug is that the second step happened too late. The registers
saved on the stack hold a bogus return value, and it's that value
which the system call eventually returns.
3 Oct 2001
I decided to profile a stretch of UML thrashing. So, I took the
2.4.10-2ac UML (which I updated to the latest stuff, and which I'll be
sending to Alan shortly), gave it 128M of memory and 1G of swap, and
let a 'make -j' kernel build run for a couple of hours.
These are the
results. All of the system calls show up as <spontaneous>.
Somehow it wasn't linked against the profiling libc. I'll try to
figure out why not.
Some highlights:
Protecting kernel memory from userspace seems to be expensive -
mprotect is the top item on the list.
wait4 is number two, which I don't entirely understand. That's the
tracing thread. It sleeps in wait, and wakes up when there's
something that needs doing, so I don't understand why it shows up,
unless it's somehow being charged for all of the context switching
that UML causes.
Then we have other low-level VM things, fix_range and
flush_tlb_kernel_vm. These manually walk address spaces to update
them. These two are unnecessarily inefficient and can probably be
knocked far down the list pretty easily.
Finally, we get into generic kernel things, which show clear signs of
heavy swapping - page_launder, swap_out_pmd, do_anonymous_page.
The first system call which shows up is sys_brk, way down the list,
followed distantly by sys_read and sys_close.
There were 312175 system calls total. The most frequently called were
sys_brk, sys_read, sys_open, sys_newfstat, and sys_stat64, not
unexpected for a kernel build.
kmalloc was called most often from load_elf_binary, select_bits_alloc,
and load_elf_interp. __get_free_pages was called most often from
handle_mm_fault, pipe_poll, and do_fork. It called _alloc_pages,
which was called most frequently from read_swap_cache_async,
do_anonymous_page, and do_wp_page.
1 Oct 2001
I discovered a new way of breaking UML. A 'make -j' kernel build
drives the load above 150, and on 2.4.10 causes essentially a
livelock. I eventually regained control by sending SIGILL to all the
processes from the host. Plus, I got all kinds of interesting illegal
instruction and bus error deaths. These were absent on -ac2, probably
because that wasn't a totally up-to-date UML, so it was missing the
most recent bugs that I added. I'm going to track those bugs down by
updating UML in the -ac tree bit by bit and seeing which bit causes
these nasty little problems.
Today I redid the signal delivery code. Now all the saving and
restoring of state happens in kernelspace rather than on the process
stack like before. This allows the task structure to be protected
from processes. Since that was the only hole in the protection of
physical memory, that is now fully protected against being changed
from userspace.
24 Sep 2001
I made a .deb and an RPM in preparation for releasing 2.4.10, and
Jacques Nilo reported that yesterday's fix wasn't enough. I had
forgot about an instance of MAP_SHARED | MAP_ANONYMOUS. So, I fixed
that, and that is 2.4.10-3um. And that is the basis of the official
2.4.10 UML release.
24 Sep 2001
I thought of an easy fix for the stack capturing problem that
prevented UML from booting on 2.2 hosts. Basically, a new process is
created which stops itself, and when that happens, the parent grabs a
copy of the stack and uses it to create a context for future threads
to run in. On 2.2, the parent used ptrace to extract the contents of
the stack from the child word by word. I looked at that code and
decided it would be much easier to map the stack MAP_SHARED so it
would be shared between parent and child and the parent could just
memcpy it to a safe place rather than ptracing it out.
What I forgot was that, while 2.4 supports MAP_SHARED | MAP_ANONYMOUS,
2.2 doesn't. So, on 2.2 hosts, UML wouldn't even begin to boot.
The easy solution was to go back to MAP_PRIVATE | MAP_ANONYMOUS, but
clone the new process with CLONE_VM, making it a thread, which allows
the parent to copy the stack directly, since they're both in the same
address space.
This fix makes 2.4.10 usable, so I've released another patch and
updated CVS.
23 Sep 2001
Linux released 2.4.10 today, so I updated UML as well. I decided not
to base this on the latest UML patch, since that it not entirely
healthy at the moment. My sigaltstack fixes broke UML totally on 2.2
hosts. So, 2.4.10-1um is 2.3.9-8um updated to 2.4.10.
CVS is not updated, but will be once I have the sigaltstack thing
fixed and that pool updated to 2.4.10.
22 Sep 2001
The last set of -ac patches went into -ac14. That brings Alan's tree
reasonably up-to-date.
I released 2.4.9-8um yesterday and 2.4.9-9um today. -8 was some bug
fixes and cleanup. -9 was fixing sigaltstack and doing a lot of cleanup
and rearrangement of the signal delivery code. This sets me up to redo
the entire signal delivery mechanism so I can finish protecting all of the
kernel's physical memory from userspace.
18 Sep 2001
That last batch of patches went into -ac12. So, the next batch is off
to him, plus one from Andrea
Arcangeli which fixes a declaration which is needed to
compile UML successfully.
Once these are in, the -ac tree is almost up to date. It'll be one
CVS release behind, which is OK because there are some tweaks I want
to make to the address space reorg. So, I'll get that right and send
it in rather than sending it in two pieces.
15 Sep 2001
I released 2.4.9-6um last night. It contains the already-mentioned
COW header changes. It also occurred to me that I can fix the
mlockall bug by sticking UML at the top of the address space where
it's supposed to be anyway. So, I went ahead and did that. This
allowed me to get rid of the vmas that UML needed to stick in each mm
to prevent mmap from reallocating areas of virtual memory that UML is
living in. This, plus the fact that these vmas had no ptes, caused
mlockall to cause major damage to UML by trying to unmap it. Putting
UML above TASK_SIZE causes it to be ignored by mmap, and the problem
just disappears. This also let me get rid of the nasty address space
reservation code that was needed in order to prevent libc from mapping
stuff in where UML wanted to put stuff.
In other news, I'm back in the ac patch business. First is a
patch that I've been sitting on
all week which defined hz_to_std and allows UML to build again. Then,
we have
calls to malloc, calloc, and free are now
converted into calls to kmalloc and kfree when the slab
allocator is running
14 Sep 2001
Greg Lonnon and I have been fiddling with the COW file header format.
I had already discovered that blindly copying the backing file path
provided by the user into the header is a problem when it is a
relative path. That COW file won't be usable by a UML run in a
different level of the directory hierarchy because, from there, the
relative path stored in the header doesn't refer to the backing file.
The fix is to write an absolute pathname into the header.
Greg had a couple of other good ideas which we thought should be
implemented earlier rather than later
The header should be able to hold a MAXPATHLEN-sized backing file name
rather than the current 256 bytes.
It should be in network byte order. This will allow COW files to be
moved between big-endian and little-endian hosts. Whether the
underlying filesystem can be mounted in UML after the move depends on
whether the filesystem has its metadata byte-swapped correctly. But,
at least the COW header won't prevent it from working.
These two are not backward compatible, so we bumped the COW header
version and made these changes in the version 2 header. The driver
can read both V1 and V2 headers but it will only write V2 headers.
The absolute pathname change is in 2.4.9-5um since it was small and
backward compatible. The other two will be introduced in 2.4.9-6um.
The uml-user list had a couple of
interesting posts from UML users today
Martin Volf did a Slackware 8.0
installation inside UML and wrote a
page describing how he did it.
Tim Robinson had some problems with the TUN/TAP transport and
posted a nice diagnosis of them.
10 Sep 2001
Been playing with the tools and website lately. I added a bunch of
new features to the mconsole
client (and promptly had to
fix it), and fixed uml_net
building on 2.2.
I also restructured the web site build somewhat to make it more manageable.
6 Sep 2001
I tracked down the process segfault problem. It was caused by a newly
forked child inheriting some pages that were swapped out, but hadn't
been unmapped. The code that it ran on its first quantum didn't
update its address space correctly, so those pages remained mapped.
Having chased that problem down, I'm releasing 2.4.9-4um with that fix
plus Chris Emerson's latest ppc changes.
1 Sep 2001
After much ado, I revamped the UML
download page. It
essentially replaces the Sourceforge project download page. I did
this in order to be able to let people select the mirror they want to
download from and to be able to put explanatory information on the same
page as the download link. If it is missing stuff that you'd like to
see, regardless of whether it's on the SF download page,
I'd like to know about it.
There are a couple things that aren't working right now - the
'Changelog's don't link to anything, and most of the SourceForge root
filesystem links don't work. I'm in the process of copying the
filesystems over to SF to fix this.
It's now pretty trivial for me to add mirrors, so if you have a box
available (particularly if it's in a part of the world not
well-covered by the UML global mirror system),
let me know.
30 Aug 2001
Thanks to what looks like an all-night debugging session on the part
of Yon Uriarte, the TUN/TAP backend now works. You'll need the latest
uml_net for this. It wasn't setting IFF_NO_PI, which was causing
extra cruft to be stuck on the front of the packet, which probably
required the broken nastiness I had to add to the driver. Adding that
and backing out all the skbuff fiddling made everything work a lot
better.
So, I released 2.4.9-3um with the fixed driver in it, plus new entries
in config.release, defconfig, and Configure.help.
28 Aug 2001
I implemented a TUN/TAP backend for the network driver. It involved
more work than I expected. A lot of it was due to restructuring other
code in order to keep the code relatively clean.
I haven't done any stessing or timing of it, but I did happen to
notice that pings over TUN/TAP are about 10x faster than pings over
ethertap. The absence of the helper handling each packet on the way
to the kernel is no doubt a big piece of that. At some point, I'll do
some bandwidth measurements against ethertap to see how much better it
is. Hopefully a lot.
26 Aug 2001
UML development took a bit of a break while I got busy with other
stuff.
In UML news, I started work on my ALS paper, got a first draft
ready, and sent it off for review. I also did a bunch of web site
work. I've been letting things fall behind for lack of time to deal
with them, so I decided to swallow my pride and start asking for
help. This necessarily involves describing what needs doing, so I
wrote most of it up, and the results are
here,
here,
here,
here, and
here.
I also made a pass over the site, fixing a bunch of hopelessly
outdated and wrong things, and probably leaving some things which are
only moderately outdated and wrong.
16 Aug 2001
I've been having fun playing with crashme. It's a great little tool.
It generates buffers full of random data and then executes them. It
runs differently on UML than on the host, which it shouldn't. The
problems I've tracked down so far are signal handling bugs. UML
wasn't handling write faults correctly when the accessed memory was
readonly, and it wasn't properly segfaulting processes to which
signals couldn't be delivered (because their stack pointers were
garbage). This last was the bug I was chasing a couple days ago.
There are still problems. The first process (crashme +2000 666 100)
runs just as it does on the host, but the next one (crashme +2000 667
100) doesn't. On the host, the segfault handler somehow gets bus
errors in libc, which I don't understand, and that doesn't happen
under UML.
On IRC yesterday, Lennert Buytenhek clued me in on how to reliably
segfault processes and crash UML. He was running 8 "du /". That
didn't work for me, but 16 of them does. The segfaults are on pages
that are mapped in but shouldn't be (their ptes say that they should
be mapped out, and somehow that didn't happen). So, those pages were
presumably allocated for something else, and contain garbage from the
perspective of the process that should have unmapped them, and so it
segfaults.
The panic looks like memory corruption. I turned on slab debugging,
and it looks like that makes the panic go away.
Well, Linus released 2.4.9 today, so it's time for me to go into my
UML release routine. When I did the obligatory kernel build on 2.4.9,
one of the crashme fixes turned out to be bogus. It was doing the
segfault-during-signal-delivery check too early, so it caught fixable
segfaults that happened because the stack needed extending or was
readonly.
14 Aug 2001
2.4.8-2um is out as of yesterday. I made the freshmeat announcement
of 2.4.8 this morning.
I chased the crashme bug a little. Somehow, a signal is marked as
being pending, but it's never actually delivered and reset. So, no
further signals can be delivered to that process from then on. This
makes it unkillable and unstoppable.
I also took the first step towards making UML secure against nasty
users. UML physical memory, except for the task structure and kernel
stack, are protected from userspace access. I still need to protect the
task structure and kernel virtual memory. The task structure is a bit
tricky because of the signal delivery code. It runs on the process
stack and is considered to be userspace code. However, it needs to be
able to modify the task structure to restore state that it saved
before the signal delivery. So, if the task structure isn't writable,
this isn't possible. Further thought on the subject is necessary.
13 Aug 2001
Well, Linus released 2.4.8 just as I was heading up north for a
weekend of camping and climbing mountains. He does this on purpose.
He released 2.4.3 when I had just arrived in San Jose for the Kernel
Summit.
Anyway, this was a relatively simple patch. It just dropped in and
worked, except that hostfs was already broken. My calculation of the
stat64 inode field was wrong. It looked at the kernel version to
decide what was in the userspace headers. I discovered the error of
my ways when I booted up a Debian UML to produce the 2.4.8 .deb. This
is a 2.2 filesystem (with .st_ino in stat64) with a 2.4 kernel (which
implied .__st_ino in stat64). hostfs did not build. I changed the
Makefile to just grep the appropriate header instead.
So, this fix will be the substance of 2.4.8-2um.
9 Aug 2001
The remaining differences between my pool and the ac tree are a couple
of patches that didn't go in for some reason, cleanups of printks and
some includes.
Daemonizing UML does work. I just checked it, and the only case where
it does something strange is if you background it without nohupping it
and log out. The tracing thread dies from the SIGHUP, but all the
other threads survive.
I released 2.4.7-5um today. It contains a few recent patches from
other people. I figured out how to turn -fno-common back on. I tried
all kinds of linker tricks to throw errno.o out of the binary. Then I
discovered that the linking that had already taken place had destroyed
any notion of what objects anything originally came from. So,
instead, I added -Derrno=kernel_errno to all the kernelspace gcc
lines, which translates all the kernel uses of errno to kernel_errno,
and leaves the libc errno alone. That's actually a better solution
than throwing out one of the errnos because that would leave open the
possibility that the kernel and userspace uses of libc could step on
each other. Now that they're using different symbols, that's not a problem.
7 Aug 2001
Yesterday's patches are off to Alan.
In other news, daemonizing UML seems to be broken again. Grrr. That
seems to break now and then for no apparent reason.
ac9 is out with my patches in it. So, time to make the final diff
between Alan's stuff and mine to get him totally caught up with me.
6 Aug 2001
Yesterday's patches are in ac8. So, two more patches will bring the
ac tree completely up to date:
A network driver update which
adds the ability for the drivers to tell the helper about any IP
address changes. This allows the host configuration (routing and
proxy arp) to stay in sync with the interface address changing inside
UML. If you're in the habit of getting UML from the ac tree, you'll
need the latest uml_net in order to use the network when this patch
goes in because it makes an incompatible change in the helper interface.
Another batch of (surprise!) miscellaneous
fixes , including some cleanup of stack permission setting, the
apparently gratuitous locals that are needed to pursuade -pg to work
properly, a couple of symbol exports for GFS, a fix that ensures that
the pid file contains the correct pid, and yet another squashed warning.
With these in, I'll be able to diff the ac tree against mine to see
what divergences there are. I know there are some, because I
occasionally see patches fail to apply because of context conflicts
which shouldn't be there. So, there will be one more patch to clean
those up, and Alan will be completely in sync with me.
5 Aug 2001
Patch time again. This time I'm making them up ahead of ac7 coming
out. So, we have
A hostfs update , which brings
the ac tree completely up-to-date. Normally, I bundle a couple of cvs
updates into a small number of patches and send them off to Alan.
With hostfs, I decided to give him the latest stuff, since there have
been a bunch of changes spread over a number of cvs updates. This is
fairly easy since hostfs is a completely self-contained piece of code.
A network driver update , which
fixes a crash and makes net devices pluggable via the mconsole.
There's some restructuring and cleanup in this patch. Also, mconsole
actions move into keventd context from softirq context. This is
because alloc_netdevice does a GFP_KERNEL kmalloc, which has to be
done in process context.
Yet another batch of miscellaneous
fixes , including renaming CONFIG_IOMEM to CONFIG_MMAPPER,
some cleanup in the ubd driver, and removal of a number of warnings.
The complete merge of the ppc
port , which reorganizes the headers somewhat. For some
headers, there are now header.h, which is a symlink to
header-$(SUBARCH).h, which includes header-generic.h and is allowed to
do whatever it wants before and after. This provides the flexibility
needed to do things like undef stuff after the include and rename
things beforehand.
These all will bring Alan up to my 2.4.7 release, except for hostfs,
which will be completely up to date. Since I'm up to 2.4.7-4um, and
2.4.7-2um was just a hostfs fix, I might be able to bring the ac tree
up to date with one more set of patches.
Alan released ac7 this afternoon, as I prophesied, so those patches
are off to him. I'll be looking for them in ac8.
I failed to resist temptation. I looked at the diffs between the ac
tree once those patches are in and my current stuff and I noticed a
big wad of documentation. So, I rolled that up and sent it to Alan.
4 Aug 2001
OK, I'm back in the business of sending Alan patches. I sent in a
small patch which fixes the
things that broke when 2.4.7 came out. So, UML now builds and works in
the -ac tree again. It made 2.4.7-ac6 an hour or so after I sent it over.
Also, in the interest of getting the ac tree more caught up with my CVS, I
sent Alan a batch of fixes which bring him up to 2.4.6-4um:
umid fixes from Henrik Nordstrom
which create a directory based on the umid rather than having that be
the pid file. The pid file and the mconsole socket are now in that
directory.
Another batch of small fixes -
a Makefile fix, mconsole cleanups and an update to create the socket in
the umid directory.
Some config changes , also
from Henrik Nordstrom. These change the network config names to be
more explicitly UML-specific. The config.in is also cleaned up so
that it resembles the i386 config more closely.
Greg Lonnon's example iomem
driver , plus a couple of generic UML fixes that were needed
in order to make it work.
A uaccess fix which required
a surprising amount of surgery to fix. The copy_{to,from}_macros
previously regarded a fault location of 0 as meaning that the copy has
succeeded without faulting. When the address passed into the kernel
was NULL, this of course broke badly. It had a very interesting
side-effect in the case I saw. After running the command that
exercised the bug, every command on the system started failing to
start because libc was corrupted. This was something of a
head-scratcher. I eventually figured out that I was causing the
command to open NULL, the fault went undetected, and the buffer that
was supposed to have had the filename copied into it had
the filename of libc in it from a previous use. So, libc was opened
for writing with fairly severe results.
3 Aug 2001
The deb build problem turned out to be me accidentally redefining VERSION
in the upper layers of the build process. That value overrode a VERSION
in the kernel build, which resulted in a totally bogus KERNELRELEASE, which
confused a macro which tested it badly enough that it broke the build.
Simple to fix once I figured it out.
I discovered another hostfs bug on my way back from OLS. ls didn't work
and I found two bugs as a result. The easy one was that hostfs_readdir
was filling in the directory inode rather than the file inode for every
directory entry it passed back to vfs. This was fixed by having read_dir
pass the inode back up so it could be use to fill in the entry properly.
The more interesting one is that there was a source-incompatible change
made in the stat64 struct between 2.2 and 2.4. The st_ino field changed
its name to __st_ino and a new st_ino field was added at the end. The
inode appears in the same place (the st_ino/__st_ino field) making it
binary compatible. So, after changing to use the 2.4 field name (and
breaking hostfs on 2.2), I changed the hostfs build to figure out what
name to use and passing that in on the compile line to hostfs_user.c.
In other news, we (me, Rodrigo de Castro, and Livio Baldini Soares) have
decided that -pg support in gcc is broken in multiple ways. rcastro and
livio complained a couple weeks ago about UML's gprof support not working.
I finally had a look at it, and found that it was broken, but not in the
way they described.
UML crashed in a very inconvient place, and when I finally got in there
enough to figure out what was happening, it turned out that mcount was
segfaulting when it dereferenced ebp because ebp was NULL. The reason
for that turned out to be that in some procedures, mcount is absolutely
the first thing they do. Everything else calls mcount after the new
stack frame has been set up and ebp has a valid value in it (the old esp).
When the procedure is the main procedure for a thread, then ebp turns
out to be NULL.
The difference between the two sets of procedures seems to be that the
good ones have local variables and the bad ones don't. So, to work
around this bug, I added a useless, but non-optimizable, local to the
affected trampolines.
Having done that, rcastro and livio were still complaining about UML
crashing. So, I looked at it with rcastro using gdbbot (and livio did so
later and discovered the same thing). -pg was trashing edx for some
reason. A constant (which varies from procedure to procedure) is
dumped into it. This suggests that it's used for the profiling
bookkeeping somehow, but looking at the assembly, we don't see how.
mcount carefully pushes it and restores it, which is not typical of
something that is going to be used for something. The problem is that
FASTCALL procedures (which are regparam(3)) pass arguments in eax,
edx, and ecx. So, dumping this constant into edx trashes the second
argument to the procedure. A workaround for this bug would seem to be
to disable FASTCALL (and I guess that gprof support stopped working
when I enabled FASTCALL to fix a different bug).
I released 2.4.7-4um today. The main new thing is that you can change
the IP address of a ethertap eth0 device and the host configuration
will change to match. This required a bit of infrastructure which I
wanted for other reasons. The uml_net interface is now versioned,
which I've been meaning to do for a while. uml_net now goes away
cleanly when UML is killed messily. Before, it would hang around,
occupying the tap device, and when UML was rerun, the new uml_net
would emit non-intuitive error messages.
I also made hostfs build and run again on 2.2 with a bit of Makefile
hackery.
28 Jul 2001
That hostfs problem turned out to be different than I thought. Livio
Soares started chasing the problem and found that the hostfs_user
close_file didn't actually close anything. It took a pointer to a
file descriptor and closed the pointer (or at least tried to) rather
than the descriptor that it pointed to. Fixing that made hostfs
behave a lot better.
Having fixed that, I finished the page cache work for UML and it can
now successfully do the deb build through hostfs without getting the
md5sum mismatches it was getting before. Having said that, I've
started seeing a compilation problem when building UML through hostfs
that I don't get on the host.
On to OLS. My talk was in the second slot of the first day, which was
nice. It's good to get your talk over early so you can do the rest of
the conference without worrying about it. It went pretty well. I had
hoped to fit a demo in at the end, but the talk basically went the
full 90 minutes. So I did a real short watch-it-boot-up demo
afterwards while most of the crowd was filing out of the room.
There was a talk on porting Linux to the i-series IBM boxes (aka
AS-400) which was fairly interesting. They ported Linux/ppc to a
hypervisor running on OS-400, making it fairly similar to the UML
port, being a port to an OS rather than to bare hardware. Dave
Boucher, who gave the talk, made a number of comments comparing it to
UML, which was nice. He also grabbed me during lunch today to quiz me
about the COW ubd driver. It turns out that he can't do that so
easily because OS-400 doesn't have sparse files, so he can't drop
blocks down in the same location in the COW file as in the backing
file because that would allocate space. I suggested a block directory
instead of a bitmap at the beginning of the COW file and dropping
changed blocks down sequentially, but he seemed unconvinced for some
reason.
A number of people told me either they or people they knew were using
UML for various things. The FreeS/WAN project as a whole seems
extremely interested in UML for running tests on their stuff over a
virtual network. A PPPoE maintainer complained about the ethertap
transport not being intuitively obvious on 2.4. And there were a
bunch of other people who were less specific about what their interest
in UML was who were either using it or were intending to.
In other news, I discovered that the mcast network transport didn't
work when the box had no ethernet card in it. Being at OLS, I showed
this to Harald Welte and we stared at the code a bit, then asked Andi
Kleen about it. The underlying problem turned out to be that there was
no route to any multicast address because there was no interface on
the system that supported multicast. The fix seems to be to add
multicast support to the loopback device, preferably, and if that's
not possible for some reason, to the dummy device.
22 Jul 2001
2.4.7-1um is released. A change which made kernel threads sychronize with
the parent at startup caused a hang at boot. The cause was a long-standing
bug which caused initdata not to be shared between processes. Andrea
noticed the problem as well, and found the fix.
That bug was fixed and I released everything. I released it with a fairly
big hostfs problem that I didn't notice until the middle of the release
process. I changed how it opens and closes files, with the result that it
closes them later than it used to. So, it isn't too hard to get hostfs
very confused by running UML out of file descriptors.
21 Jul 2001
2.4.7 appeared yesterday. I'm looking it over to see what's new. One
interesting thing is that Alan is sending over some bits of UML which
change the generic kernel. These don't affect anything besides UML,
so they're harmless. On the other hand, they eliminate some generic
files from my patch, which is nice. It makes the UML patch appear
purer.
16 Jul 2001
Yesterday's patches are in 2.4.6-ac5. So, time to send in another
batch. This will get him up to my 2.4.6-2um. Today's batch contains
another batch of
random fixes and Greg Lonnon's
ubd COW
patch. See this
page for more information on the ubd COW driver.
I also checked in all the userspace stuff, including the deb builder,
recent changes to the tools, and the website, which I hadn't checked
in for quite a while.
15 Jul 2001
Those two pesky patches finally made it to Alan OK and were included
in 2.4.6-ac4. This gets the ac tree up to 2.4.5-8um. The next batch
will bring him up to 2.4.5-10um. It includes a bunch of
miscellaneous
fixes, the first merge of the
iomem patch,
and an mconsole
update which makes gdb and the ubd driver hot-pluggable and
runs mconsole stuff from a tasklet rather than inside the interrupt.
With some more symlink abuse, I merged the last of Chris Emerson's ppc
port patch.
14 Jul 2001
Two of the three patches I sent to Alan were broken again. However, I
figured out why. My devious little mail reader was breaking lines
when it sent out the mail, which was way too late for me to eyeball it
to make sure it wasn't messing up. Turning off this behavior results
in much better patches at the other end of the line.
I played with the UML .deb builder and got a workable .deb out of it.
I think I figured out why the process gets a checksum error at the
very end - it's on hostfs, and hostfs reads through the page cache,
but doesn't write through it. I'll have to check with a filesystem
guru on this, but it sounds right to me. Putting the process on a
normal block device results in good checksums.
13 Jul 2001
I put out a couple more patches. Highlights include
Thanks to Simon Blake, I tracked down an interesting bug last night.
The UML build turns off __i386__ in order to throw out some very
hardware-specific code that UML definitely doesn't want.
This also turns off the i386 definition of FASTCALL, which invokes an
in-register parameter passing convention that gcc supports. This
wouldn't be a problem, except that UML borrows code from the i386 port
which assumes that this convention is being used.
In the case that I was looking at, rw_down_write_failed() was getting
its semaphore address from the wrong place and using a random
userspace address as its semaphore. This could cause all kinds of
interesting side-effects, like kernel corruption from two threads
using two different random addresses as the same semaphore or process
memory corruption from the kernel writing semaphore stuff into its
memory. Hopefully, this fix will eliminate some of the strange
crashes that people ocassionally see with UML.
Here is a more
detailed description of the bug and its side-effects.
I sent the two broken patches (the
mconsole
and 64-bit
patches, see the 30 Jun 2001 entry for
descriptions) to Alan again. Hopefuly they aren't broken this time.
Also, I
fixed
a few build problems that turned up lately.
7 Jul 2001
I integrated Greg Lonnon's ubd COW patch today. It allows multiple
UMLs to share a filesystem
read-write by storing the changes in a
private file. This private file can be considered to overlay the
read-only shared file. All writes go into the private file, and reads
come from the private file if it has a valid block and from the shared
file if not.
This allows a huge savings in disk space for people running many UMLs
with large filesystems. It probably will help performance, since the
caching requirements on the host are similarly reduced.
4 Jul 2001
Two of the last three patches I sent Alan somehow got corrupted. I
suspect that what happened was I added spaces accidentally while
reading the patch in my mail composition window by trying to page it
with the space bar, then messed by the patch when deleting the
spaces. So, I'll send them in again.
Rik van Riel has been visiting for the last couple of days. He was in
Boston for Usenix, and was visiting
EMC and
MCLX
(where a number of my former coworkers from DEC now work)
after the show. Since I live a couple of hours north of Boston, I
invited him up. In doing so, I acquired the responsibility of getting
him to Logan airport at the same time that 2M people were going into
Boston to see the 4th of July fireworks and concert. I ended up
putting him on a bus that ran from outside the city straight to the
airport. I haven't heard anything from him since, so I suppose that's
good news.
2.4.6 was released last night. It turns out to be a piece of cake.
The well-known softirq fix is the only thing that needed changing. I
stuck that in, and it built and ran through my tests without a
problem. The patch is released, and I'll probably finish the rest
tomorrow.
30 Jun 2001
I sent Alan patches which will bring him up to 2.4.5-8um, which include:
a collection of small fixes ,
including ^S/^Q support for the console, some ubd driver cleanups, and
the TASK_UNINTERRUPTIBLE fix
Lennert's reimplementation of the 64-bit
file support - the first try used libc's magic support for
popping the 64-bit interfaces under the 32-bit names. That broke UML
modules badly. This version explicitly uses the 64-bit interfaces and
seems a lot healthier.
Lennert's management
console patch. This version has support for getting the
kernel version, halting and rebooting the system, and turning the
debugger on and off.
Last night, the f00f bug was bugging me, so I fixed it. It turned out
that the tracing thread was routing SIGILL and SIGBUS incorrectly.
Fixing that causes f00f to SIGILL properly.
29 Jun 2001
I found and fixed the TASK_UNINTERRUPTIBLE hang last night. It turned
out to be caused by an interrupted write in the block driver. The
driver didn't check the return value, so didn't notice that an IO
request it sent to the IO thread didn't go anywhere. That shut down
the disk IO system, which ultimately results in the whole system being
deadlocked waiting for IO that's never going to happen.
That, plus a few other things, are checked in as 2.4.5-11um.
In other news, Bill Stearns, who's always looking for more devious
things to inflict on UML, happened across the
Linux Test Project
and decided to run it on UML. UML did pretty well. There were three
failures, two of which also fail on the host. The other is the f00f
test, which causes UML to hang. I applied the obvious fix of relaying
SIGILL from UML to the process. That fixed the hang, but after a long
pause, the test's SIGILL handler apparently gets called twice.
26 Jun 2001
Those last two patches made it into ac19. Time to start thinking
about bringing the ac tree up to -7um.
I finally got Greg Lonnon's iomem match into UML. This allows a
process outside UML to communicate with one inside (or with a UML
driver) through a mmapped file.
I've also been chasing the TASK_UNINTERRUPTIBLE hang that a few people
have been seeing. It happens most easily under UML, apparently. I'm
using a recipe discovered by mistral to reproduce it (two infinite
loops each diffing two kernel pools). The longest it's taken to
reproduce is about 30 minutes. It hung on boot once. The others have
been in the 5-10 minute range.
I had a long chat with Al Viro last night with him telling me what he
wanted to see from gdb and me providing it. He ended up being puzzled
about what was happening. Following a suggestion from Daniel
Phillips, I've started instrumenting buffer_heads and pages to see
what happened to the ones involved in the hang.
25 Jun 2001
It's -ac patch time again. I boiled the -5um to -6um changes down to
two patches:
a miscellaneous fixes which
adds some IP address sanity checking to the ethertap backend, fixes
a couple of process signal delivery races, cleans up the associated
thread data a little, fixes a swap bug (which caused swapped-out pages
to never be unmapped from their processes), and gets rid of the last
vestige of the mm_changes code.
a timer patch which attempts
to eliminate missing clock ticks by never disabling the timer and
keeps track of ticks which happen when it's not safe to call the timer
IRQ. This improves things, but it doesn't eliminate missing ticks
under load.
22 Jun 2001
Some time around ac16 or ac17, someone added a call to
linux_booted_ok() which the ports have to implement. So, I sent
the patch to Alan today.
And a short bit later, I got a reply saying not to bother. The
linux_booted_ok thing was a temporary test that's going to be
removed. So, it won't appear in my pool, but if you absolutely want
to run the ac16/ac17 UML, apply that patch.
I spent the better part of the afternoon in IRC trying to figure out
the hang that mistral is seeing. No joy, but I did learn more about
the problem. I'll attack it again later.
21 Jun 2001
gdbbot got its first test yesterday when I looked at the problem that
Chris Emerson is having with UML/ppc hanging during boot. I didn't
find the problem, but was able to check that signal delivery (which
was what I thought was broken) was working fine. The next step will
be to do a post-mortem on the hang.
20 Jun 2001
I wrote a IRC gateway for gdb. This allows a gdb (like the UML kernel
debugger) to be controlled from an IRC channel. The intent is that if
someone sees a bug that I can't reproduce, but want to look at, that
person's UML gdb can be attached to an IRC channel where I can poke
around and see what's going on.
I also integrated Lennert's management console patch. This is a very
low-level interface to the kernel (like the i386 SysRq interface).
The main use for it right now is to hot-plug devices. At this point,
only the ubd driver and gdb support this. So, you can add and remove
block devices from your UML without having to reboot it. You can
switch gdb in and out the same way. I will also do the consoles,
serial lines, and network interfaces at some point as well.
15 Jun 2001
The two patches I sent to Alan yesterday are in 2.4.5-ac15. Alan
horribly mangled Harald Welte's name, unfortunately.
Today was a patch bashing day. I merged in a good number of the
patches in my queue.
Today was also the (extended) deadline for abstracts for ALS2001. So,
I sent one in. This is the most explicit that I've been so far about
my future development plans for UML. So, if you want to see how wierd
things are going to get, read all about it
here.
14 Jun 2001
IBM put out a
Linux security whitepaper
in which UML gets a pretty lame mention (down towards the bottom, there's
some prose which is basically lifted from my site). Thanks to Bill Stearns
for spotting it.
I'm finally getting around to sending off the latest stuff to Alan. The
ac tree is now two cvs updates behind. The first set will be the -5um update,
which is basically
some random fixes , including an
updated defconfig, making the console xterms go away when the machine shuts
down, making a read-only hostfs really read-only, hooking up a couple of new
system calls, allowing UML to boot on hosts with a 2G/2G address space split
12 Jun 2001
Banged out a bunch of bugs. I started booting UML with 24 megs and
plenty of swap, and running a whole bunch of stuff on it to overload
it and put it heavily into swap. This turned up a couple of signal
delivery races and a swapping bug. The signal races would cause
various strange behavior. Mostly what I saw was hangs with an
infinite sequence of sigreturns. The swap bug caused pages not to be
unmapped when they were swapped out. Obviously, this is very bad.
With the help of rcastro, I fiddled my page table macros to fix this.
I'm still seeing process segfaults. It looks like pages are being
swapped out and swapped back in with the wrong data.
8 Jun 2001
I fixed a bunch of buglets, like the console xterms not going away,
readonly hostfs not being readonly, merged Harald Welte's mcast
network transport, and a few other things, and checked them into CVS.
I also checked in the tools, so everything ought to be up-to-date and
consistent at this point.
I also have the .deb build procedure working, I think. The
uncertainty is due to the fact that I think there's a hostfs data
corruption problem. My development box runs Red Hat, and I couldn't
find RPMs for the Debian tools, so I just installed them in my Debian
filesystem (apt-get rocks, BTW :-), mount the source pool inside a
Debian UML via hostfs, and run the debian build procedure there. The
problem is that the gzipped source tarball has its md5sum recorded at
the beginning of the build and checked again at the end, and they
don't match. I also ran md5sum three times in a row on that file
while the builder was running, and got three different answers. So,
it looks like I have some debugging to do there.
3 Jun 2001
True to yesterday's promise, I sent Alan three more patches
A fix for the ethertap driver
which fixes mishandling of large packets
I got the networking cleaned up enough that I'm happy for the general public
to use it. There are three host transports, ethertap, the routing daemon,
and slip. You can have the helper do the host setup for you or not. If you
do, then getting the network running is a matter of a command line switch,
ifconfiging the device, and setting routes inside UML. This is a huge
usability improvement over the previous situation.
This is all checked in, and I'm currently building 2.4.5, which I'll release
in the next day or two.
18 May 2001
I fixed the slip interface, cleaned out some unused code which had become
a portability problem, and fixed the fix for the crash caused by someone
typing at the console too soon. It is all checked in to CVS.
17 May 2001
I grabbed 2.4.4-ac11 to see if Henrik's patch was in there, and it was. So
I don't have to worry about it any more. I guess it made ac9, but Henrik
didn't get credit for it in the ac changelog.
In other news, the ethertap interface is working reasonably well. It couldn't
do HTTP until I figured out that the mtu on host tap device needed to be 16
bytes less than the UML eth0 mtu. The helper is now more helpful. In order
to talk to the rest of the world through it, you basically just have to
ifconfig the device inside UML and add a route to the outside world, and you're
done. Much better than what we had before.
13 May 2001
Five of yesterday's six patches made it into 2.4.4-ac9. The lonely exception
was Henrik's hostfs blocksize fix.
12 May 2001
I decided to clean out my patch backlog a bit. So, I merged and sent to Alan
the following patches:
most of Roman Zippel's Makefile
cleanup , minus some bugs and some unistd.h stuff that I want to
look at more carefully
Henrik Nordstrum's
one-line fix
that allows root hostfs to work when something has changed the ubd block size
the removal of
thread.starting_exec ,
which was ancient history, is now longer needed, and was causing hangs under
heavy load
a fix for a race while getting
the stack snapshot - this was seen on SMP boxes because the child gets to run
sooner than on a UP box
11 May 2001
Chris Emerson got UML/ppc booting to a shell prompt! His uml-devel post
is here . This is the first UML port,
and it showed me how to make UML portable. There aren't really all that many
non-portable things in UML, so a port doesn't take all that much code. Based
on his work, I'm going to write up a UML porting guide, which will be found
here when it's done. If that
link is dead, keep trying until I have something to put there.
In other news, I fiddled the ethertap driver backend so that the read hang
has gone. With some help from Bill Stearns, I also figured out how to talk
to the rest of my network through the ethertap device.
9 May 2001
I got the ethertap backend to the network driver working today and I
submitted it to CVS . I haven't been able to get it
to talk to anything but the host over the tap device, but it communicates
with the host just fine.
4 May 2001
My 2.4.4 fixes, except for Andrew Morton's exitcall fix, are in 2.4.4-ac3.
I wrote and submitted my OLS paper yesterday, two days late. It's also
posted on this site, as TeX
and HTML
On the network driver front, I've got the unified front-end plus the slip
back-end working. I've started working on the ethertap back-end. After that
will come the socket and TUN/TAP back-ends. This stuff is in CVS, but I
haven't updated the patch because the ethernet driver is broken, and I don't
want a bunch of complaints from people who grabbed the latest patch without
knowing what was in it.
Update: Andrew's patch made it into 2.4.4-ac5. I was beginning to wonder.
That cleans out my pending ac patches.
The last batch of patches I sent to Alan made it into ac14.
I started looking at the two network drivers today. I think it won't
be too hard to merge them. They're pretty similar, since they're both
derived from the same code base, and the differences seem to be
orthogonal. They don't seem to have done the same things in
fundamentally different ways. I posted
my impressions for the devel list to comment on.
Andrew Morton looked at the shutdown crash that
peoplestartedseeing
lately and figured out that it
was caused by /proc being unregistered before something else tried to
remove its proc entries when it was unregistered. He sent in a patch
which reversed the __exitcall order, and Henrik Nordstrom
reported that it fixed the
crash for him.
22 Apr 2001
The fixes I made on Thursday were broken. The initrd fix
introduced a name clash with a
function in hostfs, and the sleep fix made
sleep always hang . I
didn't notice because I was fixated on getting UML to boot from an initrd
image, and that wasn't obviously showing the problem.
Anyhow, I made the fixes, submitted them to CVS, updated the patch, and sent
fixes off to Alan. I hadn't sent in Thursday's changes to Alan, so the
patches are the real thing, not just patches to the patches.
The patches I sent in a couple days ago are all in ac10 by the looks of
Alan's change log.
I figured out how initrd support is supposed to work, and implemented the
necessary stuff in UML. I booted a RH initrd image far enough to convince
myself that it works.
I also figured out the sleep hang. It turns out to be a race between the
registration of the timer irq and the first time the timer interrupt calls
do_IRQ. The timer was enabled before the registration, so if an interrupt
happened in that window, do_IRQ would bail out early, leaving the irq
permanently marked as in progress and pending. This locked out all future
timer interrupts from going through the irq system, so counters would never
be decremented, and sleeps would never wake up.
I had a chat on #kernelnewbies
with Rodrigo de Castro, who's using UML
for his compressed
caching project. He understands swapping better
than I, and told me why my new pte bits were breaking it. So, I fixed
it, and swapping now seems to work.
11 Apr 2001
Sent Alan the patches necessary
for UML to build and run in his tree.
I got back a reply which said in its entirety, "ok", which I think is good.
Maybe they will make -ac5.
10 Apr 2001
UML is now in 2.4.3-ac4. I was on IRC with Alan and a bunch of other
hackers when he merged it. He looked like he was going to start
asking a bunch of embarassing questions about my locking, but he was
concerned only about one thing, and that was a special case that
didn't need locking.
Too bad it doesn't build. The patch that Alan merged was against the
Linus 2.4.3 tree, which differs in a few respects from the current -ac tree.
8 Apr 2001
Released 2.4.3 a week or so late. Blame Linus for releasing it the
night before the kernel summit officially started. We were all in San
Jose and not able to react.
4 Apr 2001
A couple more summit tidbits that I forgot to mention in my last entry:
Willy is thinking about using UML as a testbed for NUMA support. He
wants to fire up a number of virtual machines and have them hook
themselves together so they can access each other's memory through
device files. This would allow people who don't have access to the
fancy hardware to develop and debug Linux support for these boxes.
UML may appear in the -ac trees at some point. He wanted to include it, but I
had sounded fairly negative towards that in the past. What I don't
want just yet is for UML to hit the Linus tree. Alan said he doesn't
send stuff to Linus if the author doesn't want it sent, which is fine
by me.
2 Apr 2001
Back from the kernel summit. I wanted to get a feel for whether four
things that I wanted from the host kernel were reasonable. I got two
OKs and two dings. That's fine, since the OKs were the important
ones. Here's the run-down:
Userspace manipulation of address spaces : I want to be able to
create, populate, release, and switch between mm_structs. This will
speed up UML context switches, and greatly clean up that code. I
asked Linus, and he said OK to the fairly static things that I want to
do. Apparently, there are serious complications when fiddling with the
address space of another process, but that's not what I want to do.
System call interception via signals : In order to avoid the
context switching between threads involved in virtualizing a system
call, I want to have a process intercept its own system calls by
having the host kernel deliver a signal whenever it makes a system
call. The handler would be the current syscall_handler, which would
read the arguments from its sigcontext_struct. This would change a
system call virtualization from four context switches to a signal
delivery and return. I infer Alan's OK on this from my describing it
in his presence and him not objecting.
Notification when a UML thread sleeps in the kernel due to a
page fault : For the sake of cleanliness and completeness, I want to
be able to have UML know when a thread is sleeping in the kernel and
be able to call schedule when that happens. This would let UML do as
much work as possible given its state of memory residence. Alan
rejected this on the grounds that UML would be the only sane user of
this mechanism.
Full kernel preemption : This was implicitly rejected as a UML
need by Alan's rejection of the previous item. If UML is to call
schedule whenever it sleeps, the whole kernel needs to be preemptible
because the swapped-out page might be a kernel page. This doesn't at
all mean that preemption isn't going to happen. Rather, it means that
UML doesn't have a particular need for it.
Other tidbits:
A number of people consider UML a very neat hack, including Ben
Lahaise, Andrea Arcangeli, and Eric Raymond.
Alan turns out to be a UML user. For the last month or so, he's
been booting his kernels as UML kernels before booting them as native
kernels. This is in part because recovery from a totally messed up
kernel is a lot easier with UML than with a native kernel. UML is
also his ptrace test case. It apparently does things with ptrace that
nothing else tries.
Al Viro is thinking about porting UML to Plan 9. He asked me
about what it would take. He had thought through the ptrace
requirement, and I told him about the mmap requirement, which is the
next hurdle. Plan 9 apparently doesn't have mmap. He's going to
think about how to do that.
On the trip over, I did some debugging, and I also threw in some
patches. There is now a "umid=<name>" switch for providing a virtual
machine with an identifier. This causes a pid file to be created
using that name, which is something that makes controlling multiple
UMLs through a nice UI a lot easier. This file will be replaced with
a socket to the machine console that Lennert is working on.
I also implemented __exitcall, which declares a procedure which is
to be called on machine shutdown. This was prompted by the need to
remove the pid files when the virtual machine goes away. I also
converted other existing cleanups to use this mechanism.
25 Mar 2001
I checked in a bunch of changes again. Henrik Nordstrom provoked me into
making it possible to use hostfs as a root directory by sending me a patch
that did it, but which was wrong (IMHO). He did it in the same way that
nfsroot and initrd support is done, which is by adding a special block of
code to fs/super.c inside CONFIG_HOSTFS_ROOT. That works fine, but I didn't
want to annoy Al Viro. What I did instead was to add a second registration
of hostfs as a device (not a virtual) filesystem and change the ubd driver
to support being given a directory rather than a file or block device. What
happens is that when a read request comes in to the ubd driver, it is
guaranteed to be a request for the superblock. The driver constructs a fake
superblock with the directory name in it. hostfs recognizes that and claims
the mount as its own. After that, it goes back to being a normal virtual
filesystem and doesn't bother the block driver again. The involvement of the
ubd driver is a bit of a kludge, but it works well on the command line, and I
can't think of anything better besides some kind of general support for
virtual root filesystems that cover nfs and initrd as well as hostfs.
There were also a number of patches from other people: Lennert Buytenhek's
modify_ldt patch, a bunch from Greg Lonnon, one from Gordon McNutt, and
a buffer overrun patch from Henrik Nordstrom.
23 Mar 2001
I spent a few days fixing the infinite recursive context switch bug. That
was a lot more complicated than I expected. The fix involved replacing the
shadow page tables that represent the mappings on the host for each process
with bits in the pte that say whether it is up-to-date or not. These bits
are set in the little functions that change ptes and cleared in fix_range
after it's updated the mappings. Since the process page tables are per-mm
and not per-process, a mapping that was changed in a multi-threaded process
would only be updated for one of the threads. This meant that UML processes
that share a memory context also need to share a memory context on the host.
This in turn complicated exec, since it now needs to create a new host process
in order to get out of a shared UML address space. I implemented this a few
times, and on about the third try, I got something that works.
So, aside from eliminating a nasty bug, this also makes the modify_ldt fix
more useful, since it now should work properly without any extra code, and
opens the way to more efficient context switching between threads, since they
won't have to go through the remapping that processes need.
18 Mar 2001
Lots of bugs have been fixed. I got a little list of hostfs
complaints from Al Viro, which I think I fixed. hostfs is now pretty
solid. I fixed the naming problem which cropped up if you held a file
open, then moved its directory and accessed that file by its new
name. You'd get 'file not found'. This is because I stored the full
host pathname in the inode, and when you changed its name while
holding it open by the old name, the inode continued to contain the
old bogus name. This was fixed by having anything that needs a
filename walk the dentry tree back up to the root, constructing the
current filename. The other major problem was that readdir didn't
work, resulting in missing files when a directory was copied. These
are fixed. What remains is to get rid of some interfaces which will
complain about not being implemented.
The signal delivery race is fixed. That induced me to clean up a lot
of old, crufty code in the kernel entry and exit paths. That's
sensitive code, and a few bugs in it caused some very selective and
very strange behavior.
I've put together an RPM for UML just in time for the April Linux
Magazine to hit the streets with my article in it. This is good,
because the article claims that RPMs are available, which they weren't
at the time that I wrote it. This also goes some way towards
simplifying the network mess. The RPM installs the umn_helper, which
lets the umn device run without any help from the user. It also
installs the eth tools, which are otherwise hard to find unless you
pull them from cvs.
25 Feb 2001
CVS update today. I fixed a few bugs and cleaned up a bunch of
things.
I've started keeping an up-to-date TODO list. This will help
me not forget anything important. I post it to the -devel list
occasionally to prompt people to send in whatever gripes they have.
24 Feb 2001
I'm releasing 2.4.2 today. It has a number of bug fixes and no
significant functionality changes.
A number of bugs have cropped up lately. The most significant is a
race when a process signal handler returns. There is a narrow window
in which an interrupt can cause a crash. The fix is to implement
sigreturn like the other arches and run almost all of the kernel code
on the kernel stack rather than the process stack as I'm doing now.
8 Feb 2001
I managed to reproduce a number of panics and fixed all but one of
them. The key was hitting UML with a high-concurrency ab run with
requests that fire off perl scripts which make mySQL requests, with
not too much memory, so that it is at least starting to swap.
This reproduced two bugs, one was caused by a failed memory
allocation in the middle of setting up a tracing thread request. The
failure caused a schedule, which caused a switch request, which blew
away the first, partially-set-up request. When the process was
rescheduled, its request was garbage, confusing the tracing thread
into detaching it. This was fixed by moving the allocation to before
the request started being set up.
The other one, which isn't fixed yet, is caused by the shadow page
tables maintained by arch/um/kernel/tlb.c. It occasionally needs to
allocate a page table when it sets up ptes for a new range of memory.
However, if the context switch that it's dealing with was forced by
low memory, then that allocation will fail, causing a recursive
context switch, and recursion continues until either the stack guard
page is hit, or, in the case of a kernel thread, the task structure is
polluted. I'm going to fix this by following a suggestion by prumpf,
which is to use some spare bits in the pte rather than a separate page
table to figure out what parts of the address space need updating.
And, panic number three, which is also fixed, was caused a faulty
notion of when a thread is in kernel space. The old way was to look
at whether the thread is being traced. That fails when a breapoint
was put in a signal handler before it requested that tracing be turned
off. The fix is to look at the current stack pointer. However, that
causes problems when a signal is being delivered to a process. In
this case, there is kernel code running on the process stack. So, a
flag was added to the thread structure when this is happening.
29 Jan 2001
Back from Sydney. The talk went pretty well, it was well attended,
and there was a fair amount of interest in UML there. Rik van Riel
and I wandered around Syndey until the following Friday.
While I was in .au, my OLS paper proposal was accepted, so it
looks like I'll be doing my song and dance in Ottawa this summer.
3 Jan 2001
Updated the web site with a couple pages describing hostfs and the new
console/serial line input specification.
The hostfs memory corruption problems are fixed. slab debug found
them for me. They turned out to be two string buffer overrun bugs.
I'll release a patch with the fixes pretty soon.
1 Jan 2001
I released the uml patch for 2.4.0-prerelease. You can find it
here.
I'm going to make the full release tomorrow, hopefully after fixing
the hostfs crash and getting socket inputs to work.
Today is the deadline for a first draft of my Linux Magazine article
and for OLS paper proposals. I sent them both in last night. We'll
see what happens.
27 Dec 2000
hostfs now pretty much works. I built UML from inside itself on
hostfs. I fixed some bugs in the write code, added enough mmap
support to run binaries from hostfs, and implemented statfs. However,
there is some as-yet explained memory corruption going on.
In an attempt to reproduce the MySQL problems that a couple people are
seeing, I moved some of my work, which is heavy on MySQL and perl,
into UML. I've seen no problems, which is disappointing because I'm
not any closer to finding the bug, but also nice because it shows that
it's possible to do real work inside it.
I've also been banging on the ethernet driver trying to reproduce the
server buffer overflow that I saw earlier. No dice there either.
The swapoff bug is now fixed. It turned out to be a bad idea to give
kernel threads both a non-NULL mm and active_mm. That code has been
that way for ages. I have no idea when or why it became a bad thing.
9 Dec 2000
hostfs is now almost all working. mknod doesn't work, and you can't
run binaries out of a hostfs filesystem.
I also fixed that pesky linking failure that people have seen seeing
sporadically for a while. I noticed that profiling was turned
on in the latest case that showed up in my inbox. I did a profiling
build of my own and lo! it failed to link. Since I could reproduce
it, I was out of excuses for not fixing it, and so I did. You can see
a full explanation of the problem
here
.
Those changes plus a couple of smaller ones are now in CVS. They
aren't in the latest patch because the SourceForge upload system has
been seriously b0rked. I'll update the patch when I can.
7 Dec 2000
I fixed the known bugs in the block driver. The
UML#
dd if=/dev/ubd/0 of=/dev/null
hang was due to the driver returning to the block layer rather than
continuing to process the queue when it found an out-of-range I/O
request. The dbench corruption was due to the elevator rearranging
the request queue while a request was in flight. When that request
finished, the interrupt handler was supposed to retire it by removing
it from the head of the queue. The problem is that the elevator put
some other request at the head, and that request was retired without
ever being done. Meanwhile, the original request was pushed back in
the queue somewhere, and it got done twice.
Dan Aloni has started the Windows port. He got most of the
kernel to compile. There are a number of undefined symbols from files
that don't compile yet. Overall, though, it's looking pretty good.
30 Nov 2000
I updated the site a little. The major changes involved the "ARCH=um"
build change. The
compilation
page is now
very explicit about that and there's a
FAQ
entry for it.
I fixed up the block driver a little. In the past, if you did
UML#
dd if=/dev/ubd/0 of=/dev/null
when it ran off the end of the device, it could apparently hang. This
is fixed. The problem with dbench is not fixed, but I made the
driver's synchronous mode accessible from the command line with the
"ubd=sync" switch. In sychronous mode, the driver has no problems
with dbench.
18 Nov 2000
I've got hostfs starting to work reasonably. ls now works, you can cd
around and cat things. You can't write anything, create files, or
execute them yet.
17 Nov 2000
After a bit of a hiatus, I did a CVS update. A number of buglets
relating to running UML as a daemon were fixed. The build was cleaned
up - I had hard-coded "gcc" instead of "$(CC)" in my Makefiles, the
top-level Makefile is now able to do native and user-mode builds, and
I cleaned up the drivers and fs Makefiles so that they let Rules.mk do
all the hard work.
I'm also back on hostfs. I fixed the mm problems that it uncovered.
It can now do ls on the top-level directory.
1 Nov 2000
Linus finally released the final test10 yesterday, so I made my
release last night, with a freshmeat announcement this morning. The
stack overflow problems in test9 are fixed by doubling the stack
size. There is also an inaccessible page between the two stack pages
and the task structure, so there shouldn't be any task structure corruption.
There were a number of other fixes. At the last minute, I found and
fixed a nasty race which resulted in the kernel tracing its own system
calls, resulting in some nasty stack corruption which made it hard to
figure out what happened. UML can now run when its main console is
not a terminal (i.e. /dev/null). That didn't work because it flipped
the terminal between raw and cooked mode, complaining via printk if
the ioctls failed. That led to an infinite recursion of printk
error messages which ultimately resulted in a segfault. I also made
it possible to mount host devices again. That was broke when I made
the block driver check IO requests against the device size so it could
report errors for out of bounds IO. It turns out not to be possible
to get the size of the media behind a block special file, as far as I
can tell. So, as far as the block driver was concerned, block devices
had zero size, and all IO was out of bounds.
I also started work on the hostfs filesystem. This is a virtual
filesystem which provides access to the host filesystem. The theory
is straightforward - vfs calls are converted into the equivalent
system calls on the host - but this uncovered a subtle memory
management bug. If a libc routine which mallocs memory is called, and the
break is increased, that extra memory only exists in that process. If
the kernel in another process tries using that memory (or tries
calling malloc at all), it will fault. What needs to happen is for
the context switching code to see if malloc has increased the size of
the data segment and map the new memory into the newly running
process. This also raises some SMP issues because when the new memory
is mapped in, the other processors will need to be told about it so
they can also map it. The same is true of the kernel's virtual memory.
20 Oct 2000
At long last, I added a
page for related
projects and other interesting links.
In other news, it turns out that Michael Vines wrote a Linux
executable runner for Windows that does what a UML port to Windows
would have to do and he has GPL-ed it and made it available for anyone
who wants to incorporate it into a UML Windows port. See the
todo page for a link to his stuff.
17 Oct 2000
Back from ALS. The talk went pretty well. I'll put the slides up on
the site at some point.
I fixed the stack overflow problems that people were seeing. The
stack is now two pages long, with an inaccessible third page
protecting the task structure, which is on the fourth. Now, any stack
overflows will segfault rather than polluting the task structure,
making them a lot easier to debug. This is in CVS along with a few
other changes.
2 Oct 2000
Bill Stearns decided to go overboard on root_fs production. He's been
fiddling with the mkrootfs script so that it can handle distros other
than Red Hat 6.x. He's done Red Hat 7.0, Mandrake, and Immunix.
These are all now available from the project
download page . Caldera, Conectiva, and SuSE are in the works.
26 Sep 2000
SGI released a new version of XFS for test5 and I tried to apply it to
my test8 um pool, the idea being that I could play with xfs in
userspace. The patch went in ok, with some rejects that were not too
hard to figure out. After some work, I got it to build. It didn't
boot, though. There were some changes in ll_blk_rw.c that I didn't
understand, and it looks like they are what resulted in the block
device getting a NULL buffer to do I/O into.
So, maybe I'll give XFS another try when SGI gets it slightly more
up-to-date.
25 Sep 2000
I found out why the kernel debugging interface doesn't handle
breakpoints very well. Setting breakpoints results in process
segfaults, floating point exceptions, and other strange behavior. It
turns out that do_syscall stored the current register state in the
thread structure while determining whether the process was doing a
system call. If the process hit a breakpoint in the kernel instead,
then that overwrote the state that was stored when the system call was
called. When the system call returned, that bogus state was restored,
and the process was essentially teleported back into the kernel just
after the breakpoint, leading to all kinds of strange behavior. With
that problem fixed, things work much better. The kernel debugger
seems to be basically healthy, and works just like on a normal process.
While I was fixing breakpoints, I decided to see why gdb inside a
virtual machine crashes it whenever it sets a breakpoint. There turn
out to be a number of problems. First, SIGTRAP wasn't being delivered
to the debuggee when it hit a breakpoint. This made it hard for gdb
to find out that the breakpoint had been hit, and to remove it
temporarily so the debuggee could get by it. Then, it turned out that
PTRACE_SINGLESTEP wasn't implemented. This is used by gdb to execute
the instruction which had the breakpoint and stop on the next one.
There were one or two other buglets, but now that they are fixed, gdb
seems happy with breakpoints.
23 Sep 2000
So, I've been a little lax. Here's what's happened in the last few weeks:
two bugs were fixed, the reboot bug and and shell segfault bug. That's it.
1 Sep 2000
I realized that I am starting to lose track of bugs and functionality
requests, so I dusted off the project's
bug tracking
system and put everything that I know of in it. I'm also using the
patch manager
to store the fixes. The idea is that I'll put fixes there and close
them when I make a release that contains the fix.
I fixed a context-switching bug
noticed
by Lennert Buytenhek. The problem turned out to be a race while
updating the address space of the process being restarted. If the
interrupt handler needed data from the kernel's vm area, and that area
hadn't yet been updated, then the kernel would crash. The
fix was to disable signals during that period of the context switch.
31 Aug 2000
I put in
Andrea's LFS patch. While I was in there, I cleaned
that code up somewhat. That is some of the oldest code still
remaining, and it really needed some work. I also put in the fix for
the crash caused by a
module creating a kernel thread. No word yet on
whether it's the right fix, though.
Also, Laurent Bonnaud volunteered to update the ancient filesystem in
the Debian package to potato. This is very cool. It is something
that I've been wanting to do for a long time.
25 Aug 2000
I'm releasing test7 today.
My ALS2000 paper is now available as
HTML and TeX.
I redid the RH mkrootfs script. It now prompts for the info it
needs. It also works for RH6.1 and probably RH6.2, although I didn't
test that.
23 Aug 2000
Finished my ALS paper and sent it in. That's a load off my mind.
Made some more CVS checkins. This makes the various debugging options
configurable, although I haven't tested the gprof and gcov
configurations. I also added some compatibility code to make the
Debian install happier. It now recognizes that uml can have disks,
but the disk recognizer gets stuck in state 'D' for reasons I haven't
figured out yet.
Linus just released test7, so I am building it right now (right
now as I'm writing this, and not right now as you're reading it, because
I've got no idea when you're reading this). A quick check of the
patch shows no changes that I need to worry about, so this looks like
a drop-it-in-and-it-just-works patch. We'll see...
Things look good. I booted it up, and ran a few things, and
they all worked. So, I'll run the stress testers on it tomorrow, and
if that checks out, I'll release it.
21 Aug 2000
I've made the
ptrace proxy, gprof, and
gcov support configurable. Also started playing with the Debian 2.2
install. It starts up ok, which the 2.1 install doesn't. It looks
like I'll have to fake some /proc/ide entries before it will deign to
admit that the virtual machine has disks. Right now, it's punting me
into the diskless install.
17 Aug 2000
Fixed a network driver bug which caused a crash when ab was run
against it. This might also fix the ping flood problem. The fix is
in cvs. It will appear for real in test7.
15 Aug 2000
I found out what was causing uml not to boot. It turned out to be a
casting bug which was making the compiler do pointer arithmetic rather
than integer arithmetic. This was a long-standing bug, and test6
changed things so it got hit more heavily. So, assuming that it's not
too badly broken now, I'll release test6 for real.
8 Aug 2000
Revamped the website. It will be put up as soon as Linus releases
test6 and I've integrated the changes in. This is because this site
talks about stuff which isn't really going to be released until then.
7 Aug 2000
Checked in changes which make the new debugging interface
more or less work. I also added a 'debug' command line switch which
starts the kernel in the debugger, so you have control of it from the start.
There are some problems with it. Commands attached to breakpoints
cause segfaults for some reason. It also can't step across a context
switch.
I also put in Rusty's patches. They completely revamp the config
mechanism. For some reason, there also seems to be very complete
networking/netfilter converage.
5 Aug 2000
The ptrace proxy is more or less working. I've checked it in to CVS and
announced it on my devel list.
4 Aug 2000
Got a couple of patches from Rusty. I'm apparently going
to graduate to a complete port once I've applied one of them :-) It
is nice. It gives me what looks like a complete config process rather
than the one I've kludged together. He also sent in enough exports to
allow his stuff to be modular inside a UML.
I'm integrating in Lars Brinkhoff's ptrace proxy. It's partially
working - enough that I can attach to the running thread, poke around
it, set breakpoints, etc. This without needing to detach it from the
uml tracing thread. I can't ^C gdb and have it stop the kernel
wherever it happens to be. It also doesn't seem to be following
threads as one goes to sleep and another starts up. Once these work, this
will be a huge improvement in uml kernel debugging.
2 Aug 2000
Bill Stearns
pointed out a reproducable way of crashing the kernel
yesterday. It turns out that irq_save/irq_restore were completely wrong.
irq_save was enabling signals when it should have been disabling them.
This could explain a lot of the problems I saw in test4.
Linus released test5 last night, so I'm putting out the user-mode
version today. There's nothing new in it. The virtual ethernet is in
the patch, but not enabled in the binary kernels and off by default in
defconfig.
The stress testing of this kernel produced no strange happenings.
Maybe the segfaults and other stuff in the last release weren't my
fault (heh).
27 Jul 2000
Started integrating Jim Leu's virtual ethernet driver. It
basically works, but it misbehaves a fair bit. It's unclear whether
that's the kernel's fault or the driver's.
18 Jul 2000
I/O, I/O, off to OLS I go
I'm leaving OLS a bit early because I've got a hiking weekend
coming up - Carter Dome on Saturday and possibly Moriah on Sunday.
So, no one had better expect any work from me until at least next
week...
Also discovered a few more problems which I didn't see on test2 or test3:
the occasional process segfault which I've mentioned already
a devfs segfault - I did this by displaying an xterm out to the
host; when I logged out, the kernel paniced with memory corruption in
devfs
X clients sometimes can't display against a local X server - strace says
that they're stuck in select
strace also displayed their read masks as '[?]', which doesn't
seem right
Patches for these will be forthcoming when I find fixes.
14 Jul 2000
Back from SF. Not only did Linus release test3 on Tuesday (as I
discovered when I was checking things out just before leaving for the
airport), but he also released test4 yesterday. So, it looks like I'm
going to be skipping test3 and going straight to test4.
The test3 changes are pretty minor, but enough to prevent the um
test2 patch from going in. task.priority changed to task.nice, there
were some minor locking changes, devfs_mk_dir lost a parameter, and
kernel/timer.c doesn't compile because of that field change.
With those things fixed, the kernel boots fine.
I also decided to get rid of the
pid 16 (mount) - segv changing ip to 0x10025ff2 for address 0x8064000
messages that appear at boot time. They are debugging messages to
convince me that the new uaccess macros are working. But they looked
abnormal and worried people so they are now gone. Don't worry, be
happy.
The test3 kernel runs my stress tests (lmbench and a kernel build)
fine, so I'm checking it in to CVS and
announcing it on the devel list.
On to test4. The timer.c bug got fixed. Otherwise, the patch went
in cleanly. It compiles and the resulting kernel boots cleanly.
Unfortunately, lmbench segfaults. I put in some debugging code, and
lmbench stops segfaulting. On to the kernel build. That works fine.
I try a couple more lmbench runs. They work fine. Oh well.
I'll consider this releasable. Maybe someone else can find a
better way to reproduce the problem. Check this stuff into CVS, and
out goes the
announcement.
I also updated all of the downloadable stuff and
announced it.
3 Jul 2000
I fixed the double panic bug. That was caused by a stacksize limit
that was not a multiple of 4 meg. The reason that matters is that
check_range (in arch/um/kernel/tlb.c), which is used to remap address
spaces during a context switch, assumes that remappable areas and
non-remappable areas are under different pgdirs, which represent 4 meg
apiece. Non-remappable areas are areas of address space which don't
belong to the process. Kernel text, data, and physical and virtual
memory, plus the original stack, fall into this category. They are
represented by vmas in the process mm_struct, but don't have page
table entries. If check_range runs into one of these areas in the
course of looking at something else, the lack of ptes for it will
cause it to be unmapped. Since the process stack is placed just
outside the stacksize limit, if that limit is (say) less than 4 meg,
when check_range checks it for remapping, it will also run into the
main stack provided by the host kernel and unmap it. The panic
happened when a process tried to change its name, which is stored in
that initial stack.
If you see this problem, you can change your stacksize limit to a
multiple of 4 meg, or apply
this patch to
the kernel.
Those two fixes are now checked into
CVS .
Here's the devel list post describing the changes.
2 Jul 2000
UML doesn't run on recent 2.3/2.4 kernels and I figured out
why. The signal frame size increased due to some extra x86 state that needed
to be saved. UML is responsible for making sure that there is enough stack
available when it asks the host kernel to send a signal to one of its
processes. To do this, it pokes the stack (by reading and writing a word)
a little below the current stack pointer. If there is nothing mapped there,
the seg fault handler will map a page in and all will be well. The offset
that it used to poke was a hard-coded 512 bytes, which I got by looking at
the amount of stack state the syscall handler needed (312 bytes) and adding
a bit. However, it turns out that the new stack frames are much bigger than
that, so the 512 bytes wasn't enough. Fixing this makes UML run on new
kernels. If you are seeing this problem, apply
this patch to the 2.4.0-test2 pool.
I'm also chasing a bug which causes a panic like this:
Kernel panic: Double fault on 0xbffff874 - panicing because it wasn't
fixed the first time
Strange compilation errors when you build from source
As of test11, it is necessary to have "ARCH=um" in the environment or
on the make command line for all steps in building UML, including
clean, distclean, or mrproper, config, menuconfig, or xconfig, dep,
and linux. If you forget for any of them, the i386 build seems to
contaminate the UML build. If this happens, start from scratch with
host%
make mrproper ARCH=um
and repeat the build process with ARCH=um on all the steps.
Another cause of strange compilation errors is building UML in
/usr/src/linux. If you do this, the first thing you need to do is
clean up the mess you made. The /usr/src/linux/asm link will now
point to /usr/src/linux/asm-um. Make it point back to
/usr/src/linux/asm-i386. Then, move your UML pool someplace else and
build it there. Also see below, where a more specific set of symptoms
is described.
UML hangs on boot after mounting devfs
The boot looks like this:
VFS: Mounted root (ext2 filesystem) readonly.
Mounted devfs on /dev
You're probably running a recent distribution on an old machine. I
saw this with the RH7.1 filesystem running on a Pentium. The shared
library loader, ld.so, was executing an instruction (cmove) which the
Pentium didn't support. That instruction was apparently added later.
If you run UML under the debugger, you'll see the hang caused by one
instruction causing an infinite SIGILL stream.
The fix is to boot UML on an older filesystem.
A variety of panics and hangs with /tmp on a reiserfs filesystem
I saw this on reiserfs 3.5.21 and it seems to be fixed in 3.5.27.
Panics preceded by
Detaching pid nnnn
are
diagnostic of this problem. This is a reiserfs bug which causes a
thread to occasionally read stale data from a mmapped page shared with
another thread. The fix is to upgrade the filesystem or to have /tmp be
an ext2 filesystem.
The compile fails with errors about conflicting types for 'open', 'dup', and 'waitpid'
This happens when you build in /usr/src/linux. The UML build makes
the include/asm link point to include/asm-um. /usr/include/asm points
to /usr/src/linux/include/asm, so when that link gets moved, files
which need to include the asm-i386 versions of headers get the
incompatible asm-um versions. The fix is to move the include/asm link
back to include/asm-i386 and to do UML builds someplace else.
UML doesn't work when /tmp is an NFS filesystem
This seems to be a similar situation with the resierfs problem above. Some
versions of NFS seems not to handle mmap correctly, which UML depends on.
The workaround is have /tmp be non-NFS directory.
UML hangs on boot when compiled with gprof support
If you build UML with gprof support and, early in the boot, it does this
kernel BUG at page_alloc.c:100!
you have a buggy gcc. You can work around the problem by removing
UM_FASTCALL from CFLAGS in arch/um/Makefile-i386. This will open up
another bug, but that one is fairly hard to reproduce.
syslogd dies with a SIGTERM on startup
The exact boot error depends on the distribution that you're booting,
but Debian produces this:
This is a syslogd bug. There's a race between a parent process
installing a signal handler and its child sending the signal. See
this uml-devel post for the details.
It doesn't work on hosts running 2.4.7 (or thereabouts) or earlier. The fix
is to upgrade to something more recent and then read the next item.
If you see
File descriptor in bad state
when you
bring up the device inside UML, you have a header mismatch between the
original kernel and the upgraded one. Make /usr/src/linux point at
the new headers. This will only be a problem if you build uml_net
yourself.
You can network to the host but not to other machines on the net
If you can connect to the host, and the host can connect to UML, but you
can not connect to any other machines, then you may need to enable IP
Masquerading on the host. Usually this is only experienced when using
private IP addresses (192.168.x.x or 10.x.x.x) for host/UML
networking, rather than the public address space that your host
is connected to. UML does not enable IP Masquerading, so you will
need to create a static rule to enable it:
host%
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Replace eth0 with the interface that you use to talk to the
rest of the world.
Documentation on IP Masquerading, and SNAT, can be found at
www.netfilter.org
.
If you can reach the local net, but not the outside Internet, then
that is usually a routing problem. The UML needs a default route:
UML#
route add default gw gateway IP
The gateway IP can be any machine on the local net that knows how to
reach the outside world. Usually, this is the host or the local
network's gateway.
Occasionally, we hear from someone who can reach some machines, but
not others on the same net, or who can reach some ports on other
machines, but not others. These are usually caused by strange
firewalling somewhere between the UML and the other box. You track
this down by running tcpdump on every interface the packets travel
over and see where they disappear. When you find a machine that
takes the packets in, but does not send them onward, that's the
culprit.
I have no root and I want to scream
Thanks to Birgit Wahlich for telling me about this strange one. It
turns out that there's a limit of six environment variables on the
kernel command line. When that limit is reached or exceeded, argument
processing stops, which means that the 'root=' argument that UML
usually adds is not seen. So, the filesystem has no idea what the
root device is, so it panics.
The fix is to put less stuff on the command line. Glomming all your
setup variables into one is probably the best way to go.
UML build conflict between ptrace.h and ucontext.h
On some older systems, /usr/include/asm/ptrace.h and
/usr/include/sys/ucontext.h define the same names. So, when they're
included together, the defines from one completely mess up the parsing
of the other, producing errors like:
/usr/include/sys/ucontext.h:47: parse error before
`10'
plus a pile of warnings.
This is a libc botch, which has since been fixed, and I don't see any
way around it besides upgrading.
The UML BogoMips is exactly half the host's BogoMips
On i386 kernels, there are two ways of running the loop that is used
to calculate the BogoMips rating, using the TSC if it's there or using
a one-instruction loop. The TSC produces twice the BogoMips as the
loop. UML uses the loop, since it has nothing resembling a TSC, and
will get almost exactly the same BogoMips as a host using the loop.
However, on a host with a TSC, its BogoMips will be double the loop
BogoMips, and therefore double the UML BogoMips.
When you run UML, it immediately segfaults
If the host is configured with the 2G/2G address space split, that's
why. See this page for
the details on getting UML to run on your host.
xterms appear, then immediately disappear
If you're running an up to date kernel with an old release of
uml_utilities, the port-helper program will not work properly, so
xterms will exit straight after they appear. The solution is to
upgrade to the latest release of uml_utilities. Usually this problem
occurs when you have installed a packaged release of UML then
compiled your own development kernel without upgrading the
uml_utilities from the source distribution.
cannot set up thread-local storage
This problem is fixed by the skas-hold-own-ldt patch that went into
2.6.15-rc1.
The boot looks like this:
cannot set up thread-local storage: cannot set up LDT for thread-local storage
Kernel panic - not syncing: Attempted to kill init!
Your UML kernel doesn't support Native Posix Thread Library (NPTL) and
the binaries you're running are being dynamically linked to it. Try
running in SKAS3 mode first. You might be able to avoid the kernel
panic setting the
LD_ASSUME_KERNEL environment variable on the command line:
./linux init=/bin/sh LD_ASSUME_KERNEL=2.4.1
Many commands are very restrictive about what is preserved in the
environment when starting child processes, so relying on
LD_ASSUME_KERNEL to be globally set for all processes in the whole
system is generally not a good idea. It's very hard to
guarantee. Thus it's better to move the NPTL libraries away:
If you're running Debian, you might prefer to use dpkg-divert:
# export LD_ASSUME_KERNEL=2.4.1
# mount root_fs mnt-uml/ -o loop
# chroot mnt-uml
# mkdir /lib/tls.off
# cd /lib/tls
# for f in *;
do
dpkg-divert --divert --local --rename --divert /lib/tls.off/$f --add /lib/tls/$f;
done
# exit
# umount mnt-uml
# unset LD_ASSUME_KERNEL
Process segfaults with a modern (NPTL-using) filesystem
These appear to be fixed with the tls patches from Blaisorblade that
are currently in my patchset. You can apply the entire
patchset, or you can move /lib/tls in the image away, as described
above.
Any other panic, hang, or strange behavior
If you're seeing truly strange behavior, such as hangs or panics that
happen in random places, or you try running the debugger to see what's
happening and it acts strangely, then it could be a problem in the
host kernel. If you're not running a stock Linus or -ac kernel, then
try that. An early version of the preemption patch and a 2.4.10 SuSE
kernel have caused very strange problems in UML.
Otherwise, let me know about it. Send a message to one of the UML
mailing lists - either the developer list - user-mode-linux-devel at
lists dot sourceforge dot net (subscription
info) or the user list - user-mode-linux-user at lists dot
sourceforge do net (subscription info),
whichever you prefer. Don't assume that everyone knows about it and
that a fix is imminent.
If you want to be super-helpful, read
the trouble-shooting page
and follow the instructions contained therein.
You may want to create and mount new UML filesystems, either because
your root filesystem isn't large enough or because you want to use a
filesystem other than ext2.
This was written on the occasion of reiserfs being included in the
2.4.1 kernel pool, and therefore the 2.4.1 UML, so the examples will
talk about reiserfs. This information is generic, and the examples
should be easy to translate to the filesystem of your choice.
Create the filesystem file
dd is your friend. All you need to do is tell dd to create an empty
file of the appropriate size. I usually make it sparse to save time
and to avoid allocating disk space until it's actually used. For
example, the following command will create a sparse 100 meg file full
of zeroes.
Add an argument like the following to the UML command line:
ubd4=new_filesystem
making sure that you use an unassigned ubd device number.
Creating and mounting the filesystem
Make sure that the filesystem is available, either by being built into
the kernel, or available as a module, then boot up UML and log in. If
the root filesystem doesn't have the filesystem utilities (mkfs, fsck,
etc), then get them into UML by way of the net or hostfs.
Make the new filesystem on the device assigned to the new file:
host# mkreiserfs /dev/ubd/4
<----------- MKREISERFSv2 ----------->
ReiserFS version 3.6.25
Block size 4096 bytes
Block count 25856
Used blocks 8212
Journal - 8192 blocks (18-8209), journal header is in block 8210
Bitmaps: 17
Root block 8211
Hash function "r5"
ATTENTION: ALL DATA WILL BE LOST ON '/dev/ubd/4'! (y/n)y
journal size 8192 (from 18)
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..done.
Now, mount it:
UML#
mount /dev/ubd/4 /mnt
and you're in business.
Hosted at
user-mode-linux-doc-20060501/fs_making.html 0000644 0000000 0000000 00000036442 12742461304 015277 0 ustar
Creating your own filesystems
Creating your own UML root filesystems is an alternative to
downloading them from this site, either because none of these are what
you want, or they are too large to comfortably grab.
This page decribes and links to a set of projects which provide UML
filesystem builders of various types. They all require media for the
distribution that you are going to use.
This is a script that I originally wrote in order in create
filesystems from a particular version of RH media. It has since been
taken over by Bill Stearns, who has generalized and improved it
greatly. It can build bootable UML filesystems from a wide variety of
RH-based distributions. It has been used to produce essentially all
of the RH-based filesystems available from this site.
It is available
here
(and also see
http://www.stearns.org/mkrootfs/rootfs.html
). Grab mkrootfs, functions, and mkswapfs, and run mkrootfs.
It prompts you for the information you need and is reasonably
self-explanatory.
gBootRoot
gBootRoot
is a fairly general GTK-based app for creating UML root
filesystems, plus filesystems and boot disks for physical machines.
It also uses UML in order to evade permissions problems with loopback
mounting filesystems so they can be populated.
* Its IDE design allows boot and root creation methods to be added via
plugins, and some methods may be further enhanced via add-ons. Because of
this mechanism additional user-friendly features for filesystems like the
automation of X or Networking exist for some root methods.
UML Builder
umlbuilder is a nice step-by-step UML filesystem builder. It comes in
command-line and GUI variants. Not only does it let you create a
filesystem, it sets up an rc script that runs UML with the
configuration that you specified and lets you control it from the host.
rootstrap
This is a new tool by Matt Zimmerman for producing Debian UML
filesystem images. It is a work in progress, so some of the 'no's in
the table will change at some point. It's designed to produce Debian
images quickly with one command. It has a simple config file and
looks to be fairly easy to extend.
gprof and gcov support in UML is present in CVS as of
17 Apr 2001. I sent it to Alan for inclusion in
the ac tree, and it will be in my 2.4.4 release.
gprof
To use gprof with UML, you do the following
Configure with gprof support - this is "enable gprof support" under "Kernel
Hacking". You'll need to turn on "kernel debugging symbols" first.
Build UML.
Boot it up and run your tests in it.
Halt it, and the standard gmon.out file will be written out. Run gprof as
you would with any other process
host% gprof linux
and you will get the standard reams of gprof output.
Here is gprof output from
several hours of a thrashing UML. There is some commentary on it in
this entry of
the UML diary.
gcov
Using gcov with UML is similar:
Configure with gprof support - this is "enable gcov support" under "Kernel
Hacking". You'll need to turn on "kernel debugging symbols" first.
Build UML.
Boot it up and run your tests in it.
Halt UML, and every file in the kernel that has coverage data will get a
'file.da' file. In order to see what coverage of a particular file you
achieved with your tests, run
host% gcov file.c
This will produce 'file.c.gcov', which is 'file.c' annotated with line-by-line
coverage information.
Hosted at
user-mode-linux-doc-20060501/help-doc.html 0000644 0000000 0000000 00000035037 12742461304 015033 0 ustar
Help with the documentation
The source for all UML documentation is XML, with a set of tags
invented, ad-hoc, one-the-fly, by me. This is processed into the UML
web site and the HOWTO with
XSL, using
Transformiix, which is the
Mozilla
XSL processor.
The documentation source is in CVS under /doc/web. A variety of files
are located here:
XML files - the documentation source itself
Makefile - the Makefile knows how to build and sync the web site, and
how to build the HOWTO, as both HTML and text.
sgml.xsl and uml.xsl - the XSL scripts, sgml.xsl builds the SGML
source for the HOWTO, which is converted into HTML and text with the
sgml-tools utilities, and uml.xsl builds the web site.
md.pl - a little perl script which combines page.xml, which is the web
boilerpate, into a single XML document which each of the other XML
files. This allows Transformiix to combine them into an HTML file.
sgml.pl - another little perl script which combines a subset of the
XML files into a single XML document along with some other bits of
information. This combined document is then converted into the SGML
source file for the HOWTO.
In addition, there are a large number of files which are copied
verbatim into the site. These include
A number of pictures
UML slides and papers from various conferences, in various forms
A few text files
All of the UML patches which have been sent to Alan, since I reference
these from my diary
Checking it for untrue statements, particularly HOWTOs that don't work
as advertised
Looking for HOWTOs, that while technically are true, do not describe the
best, quickest, or simplest way to get the job done
The aesthetics of this site may not be the best. Improvements are
welcome. Also, there are a lack of diagrams and pictures which might
help clarify things. Contributions would also be welcome.
Improvements to the layout and organization would be good. It might
also be nice to have some kind of automatically generated news section
on the main page which lists the latest UML happenings or the latest
site updates.
This site mostly contains HOWTO-type material. If you would like to
see other sorts of things here, speak up, or write it up and send it in.
The utilities that ship with UML need man pages. I'd like XML source
that can be converted into man page source, HTML, and SGML. It's OK
if all these outputs look something like man pages. We can add new
sections to the site and the HOWTO for them.
The XML tagset being used here is probably not the world's most sane
XML schema. Improvements and simplifications would be welcome. Also,
the abstract tags are not used everywhere they should be.
The site needs an advanced networking page. This should cover things
like
Multiple UMLs networking via the host
Setting up a virtual network with one or more nodes acting as gateways
to the host
Bridging - I have a walkthrough
which needs to be incorporated
Maybe setting up IPv6
Setting up a ppp link over a UML console or serial line in order to
run what amounts to a dial-up connection to the host. Also, setting
up the serial lines of two UMLs so that one can dial up to the other.
Hosted at
user-mode-linux-doc-20060501/help-gen.html 0000644 0000000 0000000 00000042746 12742461304 015044 0 ustar
How you can help
No matter who you are, what you do, or what level of programming
ability you have, there are ways that you can help out with UML.
Non-programmers
If you don't program at all, there are a number of things that you can
do. The main one is to just use UML. Report any bugs, problems, or
rough spots that you encounter. If you find workarounds, tell us
about them, too. There are a large number of different kinds of
environments out there, and there may be assumptions accidentally
built in to UML that cause it not to work, or not to work well, in
some of them. The more people we have using UML, and the wider the
variety of situations it's used in, the more robust it will become.
You can check the documentation - you probably will do so in the
course of learning what you can use UML for. The main thing that you
can do is report any problems you find.
If you want to help in a more systematic way, you can
proofread it - I try to be grammatical and spell things correctly, but
sometimes mistakes slip through.
fact-check it - in particular, make sure that the step-by-step
procedures actually work. They occasionaly get out-of-date,
describing methods of doing things that no longer work because new
ways have been introduced and the old ways removed. They may not work
in all environments. They are mostly written up after I've got them
working on my laptop. Your machines may differ, and they may differ
enough that my procedures don't work for you. Also, I may have found a
overly-complicated way of doing something. If you know a simpler way,
you should let us know about it.
style-check it - it may be that a way of doing something is correct,
but poorly worded or confusing. If you get lost trying to do
something, you should complain about it. If you persevere and figure
out how to make it work, it would be very useful for you to go back
over the documentation and see where you got lost. Then, figure out
what it could have said that would have prevented you getting lost,
and send that in.
Documentation fixes will be accepted in any usable form, including a
general description of something that needs doing, suggested text as
just text, or fix to the XML source for the documentation.
If you want to get more involved in the documentation, it is described
here.
User-space programmers
If you want to contribute code to UML, but you aren't a kernel hacker,
there are a number of areas that need work. There are a number of
tools that come with UML that aren't strictly part of it, but which
make using it easier in some way (such as the uml_net networking
helper) or which provide access to UML functionality (such as the
mconsole client). There are a variety of enhancements that can be
made to these.
Various applications of UML are going to require tools which don't
currently exist. For example, for UML to be most useful for virtual
hosting, there need to be tools which can configure and run a virtual
machine for a new customer. People who are doing networking of
clustering experimentation with UML need something similar, except
that it needs to be able to construct a specific network of UML
virtual machines.
The UML packaging is somewhat ad-hoc at the moment. I've learned
enough about both RPM and Debian packaging to be able to put together
defensible UML packages in both formats. However, I'm sure that
they would not be considered 100% kosher by either camp. So, there is
a need for packaging experts to have a look at the UML packaging
procedures, find the mistakes, and fix them.
Contributing code to UML itself is not completely out of the question
if you're not a kernel hacker. Since UML is a port of the Linux
kernel to Linux, the lowest levels of the code, which in other ports,
are hard-core hardware-specific code, are generally fairly
straightforward userspace code. That code does stuff like read files,
talk to pseudo-terminals, communicate over the net, etc. It's
perfectly possibly for people who don't know about kernel programming
to contribute useful stuff here.
Kernel hackers
If you're a kernel hacker, you can find and fix bugs in UML. I've
listed the bugs (and some features) that I want to fix before UML V1.0
here.
There are also a number of longer-term projects described
here. These are the
post-V1 things I want to do, but I certainly have no objection to people
starting work on them now. They include things like adding SMP
support, implementing clusters of various types (first DSM-based
clusters, moving to more standard RPC-based clusters), making Linux a
better host for UML, ports of UML to more architectures and to other
operating systems, and a variety of other things.
Other contributions
There are some things that you can do to help out UML, no matter who
you are:
New applications - I have a number of applications of UML listed
here. I am but one
person and my imagination is limited. If you find UML useful for
something and it's not listed there, let me know.
Propaganda ... err, spreading the word - If you find UML useful for
something, chances are you know other people who are doing the same
thing who don't know about UML. Tell them about it. Recommend most
strongly that they start using it. If you know of web sites or
publications that would be interested in UML, suggest that they write
about it.
Donations - Other sorts of things will be cheerfully accepted
Hardware - A good rule is that anything that gives me faster kernel
builds is good. SMP kit is also good, since UML has displayed races
on SMP machines that aren't seen on UP machines. Non-x86 hardware is
good, particularly if it comes with an interesting vendor OS (i.e. VMS
or OS/400). This will help out the UML porting situation.
Money - It can be exchanged for the aforementioned toys, so it's good
too.
Anything else - Get in
touch and we can discuss it.
Hosted at
user-mode-linux-doc-20060501/help-kernel-v1.html 0000644 0000000 0000000 00000062776 12742461304 016104 0 ustar
Help with getting UML V1.0 ready
This is the current list of things I want to fix before I consider UML
to have reached version 1.
I've attached a description to each one which will provide some
guidance as to proceed with it. If you need more details, contact me
or send mail off to the uml-devel list.
My attempts to run the Debian install procedure hung UML while it was
probing disks. I never figured out what was happening.
If someone verifies that UML can run a Debian installation, I'll
consider this fixed.
Make UML build without warnings
I've been knocking these off one by one. The remaining ones are
mostly cries for reorganization. There are pieces of userspace code
in kernel files which call libc functions without there being
declarations in scope. Mostly, these things need to be separated from
whatever kernel code they're in, and moved to a userspace file.
An exception to this is the pair of warnings from kernel/timer.c. I'm
going to fix this by bumping UML's HZ to 50.
Remember the errno redefinition warning in user_syms.c during the deb build.
Make sure that each clock tick gets counted
If you run UML under load for a while, you'll notice that clock ticks
are being missed. You can see this by doing a 'ps uax' every once in
a while, and noticing that process start times are moving. I made an
attempt to fix this by never having SIGVTALRM or SIGALRM blocked and
doing some fancy flag checking to make sure that the IRQ handler is
only called if it's safe, and otherwise just bumping a count of missed
ticks. The next time it's safe, the IRQ handler will be called once
for the current tick and once for each previously missed tick,
catching the system up with the real time.
This doesn't fix it for some reason. The only reason I can think of
is that the signals are getting stacked because a second one comes in
before the first one has been collected by the handler. I have a hard
time believing that the host can be so bogged down that it can't call
the signal handler within 1/HZ seconds of the alarm happening. The
signal is going through the tracing thread courtesy of ptrace, so
maybe that's introducing enough delay to stack the signals.
Figure out the hostfs crash that Larent Bonnaud is seeing
Laurent hasn't complained about this recently. If he doesn't, I'll
just consider this fixed.
make 'gdb=pty' work
Run UML with 'debug gdb=pty' on the command line, attach to whatever
pty gdb gets, and you'll see the problem. There's some major problem
with the terminal modes that I've never figured out.
protect kernel memory from userspace
Fixing this will make it impossible for a nasty UML user to escape
from UML and execute arbitrary system calls on the host. It has four
pieces:
Write-protect kernel physical and virtual memory whenever UML is in
userspace. The exception to this is the process kernel stack, which
needs to be writable in order for a signal handler to be run on it.
This is OK because whatever the process puts there will just be
overwritten by the signal handler. This is partially implemented -
kernel physical memory is protected, except for the four pages making
up the task structure and stack. Protecting the task structure is
complicated because there is some UML code which effectively runs in
userspace which calls the process signal handler to deliver signals.
It needs to be able to write to the task structure. I think this fix
is to move that code to before the signal starts being delivered,
which will involve redoing parts of the signal delivery code.
Protecting kernel virtual memory is a matter of walking the kernel
page tables and write-protecting everything that's mapped there.
Locating and disabling all drivers that provide access to kernel
memory through interfaces such as /proc/kcore.
Fixing the access_ok macro so it rejects any attempt to fake the
kernel into changing its own memory by passing a kernel address as an
argument to a system call.
In the host kernel, adding a new personality which segfaults any
attempt to execute a SysV system call. These are not traced by
ptrace, so a process that knows how to use them is not jailed inside UML.
Figure out why the io_thread loses its parent and its connection to the kernel
This happened once, a long time ago. The system hung because the IO
thread lost its connection to UML somehow. I haven't seen it since,
so this will probably be considered fixed.
Get either Myrtal Meep or Matt Clay
to confirm (or not) that they can no longer crash UML with
Apache/Perl/MySQL. This will probably be fixed with the new network drivers.
The new network drivers probably fixed these problems. This will be
considered fixed unless people start complaining about it again.
Disable SIGIO on file descriptors which are already being handled.
This will cut off some interrupt recursion.
This is an efficiency thing and stack depth precaution. We don't want
sigio_handler to run recursively on the same descriptor because the
descriptor will be handled at the top level.
Figure out why gdb can't use fd chan (blinky@gmx.net)
One of the ioctls that sets up the terminal for gdb returns EPERM.
Figure out why repeated 'tail /dev/zero' with swap causes process segfaults
I stopped being able to reproduce this, so it might be considered
mysteriously fixed at some point.
Set SA_RESTART on SIGPROF
This will prevent SIGPROF from causing system calls on the host from
returning EINTR and messing things up inside UML.
Allow registration of sockets for panics and console output
This involves some work on the mconsole client side to support making
the requests and work in UML to add the mconsole notification to the
panic_notify_chain and to somehow get console output to the client.
My best idea on doing this is to add support for multiple channels on
consoles and support for write-only channels. Then the mconsole
client can stack a write-only channel to a socket or something on top
of whatever channel is already there.
Replace dentry_name with d_name
Cleanup - I implemented dentry_name, not knowing about d_name, which
does the same thing. One problem might be that dentry_name expects
the file to exist, which might cause something to break when d_name
returns a '/foo/bar/baz (deleted)' name.
Bind a UML unix socket to a host unix socket
This is a cute feature which would involve passing something like
'hostsock=/tmp/.X11-unix/X0,X' on the command line, which would cause
the hostsock driver to create a /proc/hostsock/X unix socket inside
UML and pass all operations and data on it through to the host's
/tmp/.X11-unix/X0. Then, you'd create /tmp/.X11-unix/X0 in UML as a
link to /proc/hostsock/X and be able to run X clients on the host X
server without needing the network up.
Dynamically allocate all driver descriptors
The static arrays in the drivers should go away and be replaced by
dynamically allocated buffers, maybe in a list. This will turn
lookups of particular devices into O(N) operations, but maybe that's
OK. Also, the transport arrays in the various transports should
similarly go away.
Make slip_tramp conditional in slip_close
slip_open doesn't always invoke the helper through slip_tramp, so
slip_close similarly shouldn't.
The real problem here is that when a gateway address isn't specified for
slip, it just doesn't work at all.
many du / causes slab cache corruption
This wasn't entirely reproducable, but I did see a couple cases of
panics with slab corruption when running a lot of du's. I fixed the
process segfault problem, and this problem has probably disappeared as
well.
Adding an eth device via the mconsole (and probably the command line)
won't necessarily configure the device that was named
The kernel does its own ordering of devices, so configuring eth1 without
configuring eth0 gives you eth0.
Implement /proc/umid
This would let UML be configured on the inside with knowledge of what
instance it is.
Fix the ubd rounding bug spotted by Steve Schmidtke and Roman Zippel.
If the number of sectors isn't a multiple of 8, then the bitmap will
be a a byte shorter than it should be.
gdb should be the last thing to be shut down so that breakpoints can
be set late in the shutdown process.
Have the __uml_exitcall handlers check to see if they need to do anything.
If UML is ^C-ed before things are set up, then the cleanup handlers are
cleaning up after things which were never initialized.
Find the race that causes the kernel to run before the debugger is running,
and which then causes a panic and segfault.
Tests that need writing
Build and load some modules to check for unexported symbols
Swap testing - qsbench, low-memory kernel build
Rerun some existing tests through hostfs
Figure out why gdb inside UML sometimes misses breakpoints.
^C doesn't work well in a busy loop in __initcall code. I've seen the
process die with a SIGINT as well as have the SIGINT arrive to a
sleeping process, panicing the tracing thread.
When setting a conditional breakpoint on __free_pages in
free_all_bootmem_core and continuing, UML hangs.
It's not a hang. The conditional breakpoint is in a long loop, and that
makes it take a while. ^C to gdb seems to be ignored for some reason. UML
sees it and passes it right along to gdb, which does nothing about it.
Figure out what to do with __uml_setup in modules. A lot of these should
end up being normal __setup, since they don't have to happen before the
kernel runs.
Figure out why UML died nastily after ^C-ing it and hitting it in userspace.
Telnetting to a console when in.telnetd is non-executable produces an
error and a crash from a sleeping process segfaulting.
Single-stepping when a signal has arrived doesn't work. gdb
singlesteps, sees the signal, and single-steps with the signal. The
signal is handled immediately, stepping the process into the handler.
The original instruction was never executed, so when gdb puts the
breakpoint back and the handler returns, the breakpoint is hit again.
The _to_user routines refer to user data without using copy_to_user.
How to cause a strange userspace segfault - run the new UML inside UML
under gdb. Put a breakpoint on create_elf_tables. 'finish', 'next',
segfault.
With an old port-helper hanging on to a port, running a UML which want those
ports causes all of the consoles and serial lines to output their login
prompts to stdout.
Assigning console and serial line devices to files doesn't work cleanly.
UML tries to register the descriptors for SIGIO. Some other mechanism is
needed for feeding the file data into userspace.
Make sure that irqs are deactivated or freed when their descriptors are
closed.
Things to audit:
Make sure that destructors exactly undo everything the constructor does
Set FD_CLOEXEC on any descriptors that don't need to be passed across execs.
Make sure any protocols are 64-bit clean - this means not using ints, longs,
etc. Also, maybe enums are bad.
port_kern.c has port_remove_dev and port_kern_free which do almost the
same things.
Make sure that return values are actually used.
skas4 things - remove the extra two context switches per system call,
make sure it compiles with CONFIG_PROC_MM off, implement the mm
indirector, move PTRACE_LDT to /dev/mm, fix PTRACE_FAULTINFO to return
trap type and err, allow the environment, argument, etc pointers of an
mm to be set.
Hosted at
user-mode-linux-doc-20060501/help-userspace.html 0000644 0000000 0000000 00000035752 12742461304 016264 0 ustar
Help with the utilities
The next version of the protocol needs to be more packet-oriented than
it is now. In order to pass an arbitrary amount of data back to the
client, we need to support replies with multiple packets and to be
able to tell when the output is finished.
The current client needs improving:
It should be able to control multiple UMLs
It should be able to detect new UMLs as they come up
Command history should work and should be saved from one session to
the next
The mconsole should be able to receive console output and
notifications of panics. It should have some sort of intelligent
interface for letting the user see or ignore the console output.
There probably needs to be a GUI mconsole client. Emacs and IRC
mconsole clients would also be cute.
uml_switch
Its protocol should be versioned.
There needs to be an output socket for each client, rather than the
single socket that exists now. This is because one client that slowly
receiving packets could cause the socket to nearly fill up, causing
packets to another client to overflow it and lose packets. In this
case, the slow UML will unfairly cause a different UML to lose packets.
It should be able to act as a gateway to the host.
It should be able to communicate over UDP sockets as well as unix
sockets
It should be able to communicate with uml_routers on other hosts
Packaging
Someone who understands how RPM and debs work should look at the UML
packages and at least figure out what's wrong with them.
Better documentation needs to be installed with them. The HOWTO
currently isn't build and installed. There also needs to be a README
installed in /usr/lib/uml which at least describes what to do with the
module tarballs.
The UML test harness should be packaged up somehow. My current
thinking is that the tarball could be part of the packages, the
install scripts would check for the presence of perl, and if it's
there, unpack and install the test harness and test cases in the same
way that modules from CPAN are installed.
And the test suite has not been worked on for quite a while. The UML
module should be updated so it can be an mconsole client. The test
suite should be turned into a real set of tests which can be run as
part of the normal build process.
gdb scripts
James Stevenson has written some handy gdb scripts which make it easy
to print to the process list and a few other things. These need to be
put in the pool, installed in the packages, documented, and expanded.
mkrootfs
The mkrootfs script needs to be able to create bootable filesystems
from Debian and Debian-based media.
It also needs to be able to construct special-purpose filesystems,
such as minimal filesystems that run a single service, such as Apache,
sendmail, or bind.
A GUI version of mkrootfs might also be nice.
The filesystem images
Now that we have the COW ubd driver, these should be packaged up
separately as RPMS and debs.
Hosted at
user-mode-linux-doc-20060501/honeypots.html 0000644 0000000 0000000 00000044247 12742461304 015373 0 ustar
UML as a honeypot
A honeypot is a sacrificial system that's (usually) made vulnerable,
and put on the net for nasty people to break into. A properly
constructed honeypot is put on a network which closely monitors the
traffic to and from the honeypot. This data can be used for a variety
of purposes
Forensics - analyzing new attacks and exploits
Trend analysis - look for changes over time of types of attacks,
techniques, etc
Identification - track the bad guys back to their home machines to
figure out who they are
Sociology - learn about the bad guys as a group by snooping on email,
IRC traffic, etc which happens to traverse the honeypot
Traditionally, honeypots have been physical systems on a dedicated
network which also contains multiple machines for monitoring the
honeypot and collecting logs from it. This is a huge logistical pain
and prevented honeypots from becoming a common network security tool.
The advent of virtual machines such as UML has made setting up
honeypots far easier. Instead of a set of physical machines, the
honeypot is now a virtual machine with the host filtering and
monitoring network traffic and collecting logs. Even better, one host
can have multiple honeypots running on it, those honeypots can be
configured in a realistic virtual network, and they can be distributed
on a CD rather than a truck.
For more information about honeypots (and honeynets) in general, see
honeynet.org.
UML honeypot support
Because of the interest from the honeypot people in UML (and support
from Dartmouth
ISTS), a number of features have been added to UML in order to
make it more useful as a honeypot.
These include
tty logging - secure logging of all UML tty traffic to the host
hppfs - a UML filesystem which allows entries in the UML /proc to be
arbitrarily rewritten from the host, making it possible to make the
UML pretend to be a physical box
skas mode - UML can operate in a mode which creates process address
spaces which are identical to the host
tty logging
A problem with physical honeypots is that it's hard to capture
keystrokes. If the bad guys are using ssh to reach the honeypot (and
they probably are since they tend to be very security-conscious),
sniffing the network doesn't help since that traffic is encrypted.
So, you need to capture keystrokes by running something on the
honeypot. This is problematic since you have to assume that it has
been thoroughly compromised, so the logging mechanism may also have
been compromised.
There are various kernel patches and other kludges (such as an
instrumented bash) to implement tty logging on physical honeypots.
They all suffer from the problem that they can be subverted or
disabled if their presence becomes known to the intruder.
UML solves this problem with a patch to the tty driver which logs all
traffic through tty devices out to the host. In contrast to the
physical honeypot logging mechanisms, this is undetectable and
unsubvertable. It causes no network traffic or anything else which
can be detected from within the honeypot. It's also in the UML
kernel, which means it can't be defeated by anything the intruder
might do.
For more information on UML's tty logging, see
this page.
hppfs
The single largest weakness of UML as a honeypot is that it is easy to
for a user to tell that it is a UML rather than a physical machine.
The best sources of information that can be used to distinguish UML
from a physical honeypot is /proc. Files such as /proc/mounts,
/proc/interruppts, and /proc/cmdline contain very UML-specific
information.
To lessen the danger of a UML honeypot being fingerprinted by an
intruder in this way, the UML /proc can be covered over by a special
filesystem which allows /proc entries to be changed from the host.
The UML hppfs ("HoneyPot ProcFS") filesystem allows /proc entries to
be added or deleted, and for existing entries to be replaced or
modified.
See this page for
information on setting up and customizing a fake /proc using hppfs.
skas mode
A major weakness of UML in security-related applications, including
honeypots, has been its overall architecture. UML has loaded itself
into the top .5G of its process' address spaces, leaving the remainder
to the process. Thus, the UML kernel, including all of its data, is
accessible to UML processes. And, by default, that data is also
writeable by UML processes. For any security-related application, for
which UML must be a secure jail for root, the fact that the data can
be written is a big problem. For honeypots, the mere fact that the
UML kernel is visible, is a big problem. An intruder can test whether
a honeypot is a UML simply by looking at the top of its address space.
To address this problem, UML was recently reworked to allow it to run
in a mode in which the UML kernel is in a totally separate host
address space from its processes. This makes the UML kernel binary
and data totally invisible to its processes, and to anyone logged in
to it. It also makes UML kernel data secure from tampering by its
processes.
This new mode (called "skas", for "Separate Kernel Address Space" -
the old mode is retrospectively called "tt", for "Tracing Thread")
therefore makes a far better honeypot than tt mode. The one
disadvantage is that it requires a patch to the host kernel in order
to run. This patch is available as the latest host-skas patch from
here.
UML must have CONFIG_MODE_SKAS enabled. It will check for the
presence of the patch on the host and use skas mode if possible. If
you see the following messages at the start of the boot log, UML is
running in skas mode:
Checking for the skas3 patch in the host...found
Checking for /proc/mm...found
As a side benefit, you will notice that skas mode, which is secure, is
noticably faster than tt mode, which is not. Furthermore, skas mode
is roughly an order of magnitude faster than tt's "jail" mode, which
is how you previously needed to get the same level of security offered
by skas mode.
If you want to access files on the host machine from inside UML, you
can treat it as a separate machine and either nfs mount directories
from the host or copy files into the virtual machine with scp or rcp.
However, since UML is running on the the host, it can access those
files just like any other process and make them available inside the
virtual machine without needing to use the network.
This is now possible with the hostfs virtual filesystem. With it, you
can mount a host directory into the UML filesystem and access the
files contained in it just as you would on the host.
Note that hostfs is currently not available on 2.5. The reason is
that there was an fs.h rework early in 2.5 which required filesystem
changes, and I haven't got around to updating hostfs to those changes.
Using hostfs
To begin with, make sure that hostfs is available inside the virtual
machine with
UML# cat /proc/filesystems
hostfs should be listed. If it's not, either rebuild the kernel with
hostfs configured into it or make sure that hostfs is built as a
module and available inside the virtual machine, and insmod it.
Now all you need to do is run mount:
UML# mount none /mnt/host -t hostfs
will mount the host's / on the virtual machine's /mnt/host.
If you don't want to mount the host root directory, then you can
specify a subdirectory to mount with the -o switch to mount:
UML# mount none /mnt/home -t hostfs -o /home
will mount the hosts's /home on the virtual machine's /mnt/home.
hostfs command line options
There is a hostfs option available on the UML command line which can
be used confine all hostfs mounts to a host directory hierarchy or to
prevent a hostfs user from destroying data on the host. The format is
hostfs=directory,options
The only option available at present is 'append', which forces all
files to be opened in append mode and disallows any deletion of files.
To specify append mode without confining hostfs to a host directory,
just leave out the directory name so that the argument begins with a
comma:
hostfs=,append
hostfs as the root filesystem
It's possible to boot from a directory hierarchy on the host using hostfs
rather than using the standard filesystem in a file.
To start, you need that hierarchy. The easiest way is to loop mount an
existing root_fs file:
host# mount root_fs uml_root_dir -o loop
You need to change the filesystem type of / in etc/fstab to be 'hostfs', so
that line looks like this:
none / hostfs defaults 1 1
Then you need to chown to yourself all the files in that directory that are
owned by root. This worked for me:
host# find . -uid 0 -exec chown jdike {} \;
If you don't want to do that because that's a filesystem image that
you boot as a disk, then run UML as root instead.
Next, make sure that your UML kernel has hostfs compiled in, not as a
module. Then run UML with the following arguments added to the command line:
If you need to build hostfs because it's not in your kernel, you have
two choices:
Compiling hostfs into the kernel:
Reconfigure the kernel and set the 'Host filesystem' option under
'Processor features' to 'Y'. Recompile the kernel and reboot it.
Compiling hostfs as a module:
Reconfigure the kernel and set the 'Host filesystem' option under
'Processor features' to 'M'. Rebuild the kernel modules. hostfs will
be in arch/um/fs/hostfs/hostfs.o. Install that in /lib/modules/`uname -r`/fs
in the virtual machine, boot it up, and
hppfs reads a proc hierarchy in the current working directory
on the host of the UML. When a UML process accesses a /proc entry,
hppfs consults the corresponding entry in this hierarchy to decide what
to do. This entry is a directory which contains a file which
describes what to do with that entry. The possibilities are as
follows:
No entry on the host - hppfs passes the operation on to the UML /proc
- effectively this means that entry is unmodified. Most /proc entries
contain no information about whether the machine is a UML, so these
can be left alone.
The host entry is a normal file or symlink - the contents of this file
(or target of the link) will replace the UML /proc entry. This is
appropriate for /proc entries, such as /proc/cmdline, which can be
replaced by the corresponding /proc entry on the host. It is also
appropriate for entries which must be changed, but which are static.
In this case, you'd drop the static file into the host proc hierarchy
for that UML.
The host entry is a directory and contains
"remove" (a normal file) - this causes the entry to disappear from the
UML /proc. This can be used to remove things which don't exist on
physical machines, such as /proc/exitcode and /proc/mconsole.
"r" (a unix socket) - this causes the entry to be generated on
demand by the process attached to the socket. This is useful for
entries which must be dynamically generated, but can be generated from
scratch on the host.
"rw" (a unix socket) - this is similar to "r", except that the current
UML /proc contents of that entry are written to the socket before the
new contents are read. This allows the host process to filter the UML
/proc data. This is useful when data must be removed from a UML /proc
entry.
The files inside /proc/pid need slightly special handling
because you can't know on the host what pids are going to be present
inside UML. UML looks in the pid subdirectory of the host's shadow
proc for all modifications of UML /proc/pid entries.
The uml_utilities tarball (under honeypot/) contains a few things to
simplify the disguising of a UML's /proc:
hppfs.pm - A perl module which acts as a server for any dynamic proc
entries that need to be generated.
hppfslib.pm - A library which contains a set of utilities for
constructing a fake proc environment.
honeypot.pl - A sample driver for hppfs.pm and hppfslib.pm which
overrides a number of UML /proc entries in various ways.
Example
Here is an example of setting up a customized UML /proc using hppfs.
First, boot a UML with hppfs enabled ('grep hppfs /proc/filesystems'
to check). Login in and mount hppfs over /proc:
UML# mount none /proc -t hppfs
ls /proc and notice that it doesn't seem any different than before:
Create a proc directory in the current working directory of the UML
host% mkdir proc
Now, put something into proc/cmdline
host% echo 'hi there' > proc/cmdline
and look at the UML's /proc/cmdline
usermode#~: cat /proc/cmdline
hi there
This should give you a good idea of how to make simple modifications
to the UML's /proc. A more realistic example would be to make
/proc/cmdline the same as on the host. You can do this by copying the
host's /proc/cmdline into proc, but a symbolic link will work, too:
host% ln -s /proc/cmdline proc/cmdline
usermode:~# cat /proc/cmdline
auto BOOT_IMAGE=2.4.19 ro root=306 BOOT_FILE=/boot/vmlinuz
With that step, you removed one way for an intruder to fingerprint the
machine as a UML. There are a bunch of others, and it would be nice
to automate the creation of realistic, fake data for them as well.
Grab the uml_utilities tarball, unpack it, and cd into honeypot/. Run
honeypot.pl with the current directory of UML as an argument:
call the hppfs server main loop, which will service requests from the
hppfs client inside UML for dynamic proc entries:
$hppfs->handler();
If you look at the proc directory now, you will see a bunch of new
entries:
host% ls ~/linux/2.4/um/proc/
devices filesystems iomem mounts uptime
exitcode interrupts ioports pid version
These all need some sort of manipulation.
Removal - /proc/exitcode is specific to UML; writing to it allows the
exit code of UML on the host to be changed. This obviously can't be
present in a honeypot, so the argument
"exitcode" => "remove",
to hppfs::add causes it to create an exitcode directory in proc on the
host:
host% ls -al ~/linux/2.4/um/proc/exitcode
total 8
drwxrw-rw- 2 jdike jdike 4096 Dec 18 14:43 .
drwxrw-rw- 11 jdike jdike 4096 Dec 18 14:43 ..
-rw-rw-rw- 1 jdike jdike 0 Dec 18 14:43 remove
The 'remove' entry tells hppfs to hide the UML /proc 'exitcode' file,
and 'ls /proc' inside the UML shows that it is indeed gone.
Fabrication - we've already seen an example of fabricating a proc
entry from scratch. Dropping a static file into proc will cause it to
replace the corresponding UML /proc entry. However, lots of /proc
files are dynamic, so looking at it twice and seeing exactly the same
data would be a tip-off that something's wrong. If the host's /proc
entry is usable, then a symbolic link from the shadow proc to /proc
will be fine. This is what these arguments to hppfs::add do:
(although the actual hppfslib implementation of proc() currently has
the hppfs server execute 'cat /proc/<file>')
Filtering - sometimes the host's /proc or anything else we can
generate on the host won't be right. Instead, we might want to take
the UML /proc file contents and manipulate them somehow. An example
is /proc/filesystems. In this honeypot, it contains an entry for
hppfs, which can't be allowed to remain. So, this argument to
hppfs::add causes that one line to be removed, leaving everything else
the same:
"filesystems" => remove_line("hppfs"),
Note the "pid/mounts" entry. That covers all references to any
/proc/pid/mounts file inside UML.
Hosted at
user-mode-linux-doc-20060501/index.html 0000644 0000000 0000000 00000052142 12742461304 014443 0 ustar
The User-mode Linux Kernel Home Page
User-Mode Linux is a safe, secure way of running Linux
versions and Linux processes. Run buggy software, experiment with new
Linux kernels or distributions, and poke around in the internals of
Linux, all without risking your main Linux setup.
User-Mode Linux gives you a virtual machine that may have more
hardware and software virtual resources than your actual, physical
computer. Disk storage for the virtual machine is entirely contained
inside a single file on your physical machine. You can assign your
virtual machine only the hardware access you want it to have. With
properly limited access, nothing you do on the virtual machine can
change or damage your real computer, or its software.
This web site
This page contains brief
descriptions of how people are using User-Mode Linux.
The kernel page has a
brief description of the kernel, its capabilities, and how it makes
use of the hosting kernel's resources.
The download page tells
you what you need in order to run the kernel and where to get it.
The running it page
describes how to run the kernel, what to do once it's running, and how
to shut it down.
For some fun with UML, see the
Sysadmin Disaster of the Month contest where you will trash a
UML system and figure out how to fix it.
This page describes how
to get the source and build it, plus how to compile and use kernel
modules.
The to-do page lists the
current major bugs, patches which would be nice to incorporate, and
lousy code that needs to be cleaned up.
Here is a screenshot
of a two-machine virtual network, with one node running a local X
server and the other displaying an xterm onto it.
Rusty Russell wrote a nice HOWTO, which can be found
here.
Also see David Coulson's
UML community site
for news, forums, and lots of other stuff.
Plus, there's lots more! Look at that sidebar on the left and see all
the stuff that hasn't been mentioned yet.
Help!
This project has outgrown my ability to run it on a volunteer,
part-time basis and parts of it have not had any attention for a while
and have fallen seriously behind. So, I'm looking for people to help
pick up the pieces that I've dropped.
There are jobs suitable for all ages and skill levels. See
Overview - the
overall picture of what needs doing, plus some miscellaneous things
that didn't fit anywhere else
Documentation - if
you like doing documentation, this describes how the UML docs are done
and lists some of the things that need doing
UML utilites -
for people who code, this page describes what needs doing with the
various UML utilities and a few other areas of userspace
UML version 1.0
- If you want UML to reach V1.0 already, here's the current list of
bugs, small features, and cleanups that I want done before I consider
it to have reached that milestone.
UML projects - If
you're a far-seeing kernel hacker, this is my list of post-version 1
projects for UML.
Examples
There are a number of pages showing the user-mode kernel in action:
You can make a donation to the UML project through PayPal, and help
ensure that I can continue to spend time on it, by clicking on this button:
Or you can send it the old-fashioned way to this address:
Jeff Dike
375 Tubbs Hill Rd
Deering, NH 03244
USA
Sponsorships
Thanks to
Dartmouth
ISTS and Bill Stearns for sponsoring the security work in UML
RidgeRun
for sponsoring the floating point ptrace fixes
The UML SMP and highmem work, and the existence of the 2.5 BK
repositories were sponsored by
Cluster Filesystems.
Thanks to Steve Freitas, Harry Zink, and the
PostNuke
Project for donating an SMP box so that host SMP problems can
be fixed.
HP contributed a
nice IA64 workstation to the UML project.
If there's a UML feature that you need or a bug you want fixed now
rather than when I get around to it, and you'd be interested in
sponsoring the work, get in touch with me at jdike at karaya dot com.
Resources
This project is hosted at
sourceforge.net,
which provides a number of useful services:
This web site is hosted at sourceforge
The project page , which contains pointers to everything associated with
the project
user-mode-linux-devel - for discussion
of the innards, code, algorithms, etc.
user-mode-linux-user - for discussion
of the outards, questions about running the kernel, configuring it,
etc.
Bug reports may be sent to either list.
Searchable list archives exist at MARC
(uml-user,
uml-devel) and gmane
(
uml-user,
uml-devel).
CVS - The code is in cvs, and can be browsed from
here.
There is also anonymous CVS, which can be accessed by
IRC (see www.irchelp.org/ for more
information on IRC) - #uml on irc.oftc.net is where I hang out 2-3
days a week. It's good for general UML questions and chit-chat.
#kernelnewbies on oftc.net is a also good place for UML questions
and discussion that relate to kernel development in general.
Send any questions, comments, flames, etc to jdike at karaya dot com
Hosted at
user-mode-linux-doc-20060501/input.html 0000644 0000000 0000000 00000045173 12742461304 014501 0 ustar
Serial line and console input
It is possible to attach UML serial lines and consoles to many types
of host I/O channels by specifying them on the command line.
You can attach them to host ptys, ttys, file descriptors, and ports.
This allows you to do things like
have a UML console appear on an unused host console,
hook two virtual machines together by having one attach to a pty and
having the other attach to the corresponding tty
make a virtual machine accessible from the net by attaching a console
to a port on the host.
The general format of the command line option is device=channel.
Specifying the device
Devices are specified with "con" or "ssl" (console or serial line,
respectively), optionally with a device number if you are talking
about a specific device.
Using just "con" or "ssl" describes all of the consoles or serial
lines. If you want to talk about console #3 or serial line #10, they
would be "con3" and "ssl10", respectively.
A specific device name will override a less general "con=" or "ssl=".
So, for example, you can assign a pty to each of the serial lines
except for the first two like this:
ssl=pty ssl0=tty:/dev/tty0 ssl1=tty:/dev/tty1
The specificity of the device name is all that matters; order on the
command line is irrelevant.
Specifying the channel
There are a number of different types of channels to attach a UML
device to, each with a different way of specifying exactly what to
attach to.
This will cause UML to allocate a free host pseudo-terminal for the
device. The terminal that it got will be announced in the boot log.
You access it by attaching a terminal program to the corresponding tty:
screen /dev/pts/n
screen /dev/ttyxx
minicom -o -p /dev/ttyxx - minicom seems not able to handle pts
devices
kermit - start it up, 'open' the device, then 'connect'
terminals - device=tty:tty device file
This will make UML attach the device to the specified tty (i.e
con1=tty:/dev/tty3
will attach UML's console 1 to the host's /dev/tty3). If the tty that
you specify is the slave end of a tty/pty pair, something
else must have already opened the corresponding pty in order for this
to work.
xterms - device=xterm
UML will run an xterm and the device will be attached to it.
Port - device=port:port number
This will attach the UML devices to the specified host port.
Attaching console 1 to the host's port 9000 would be done like this:
con1=port:9000
Attaching all the serial lines to that port would be done similarly:
ssl=port:9000
You access these devices by telnetting to that port. Each active
telnet session gets a different device. If there are more telnets to
a port than UML devices attached to it, then the extra telnet sessions
will block until an existing telnet detaches, or until another device
becomes active (i.e. by being activated in /etc/inittab).
This channel has the advantage that you can both attach multiple UML
devices to it and know how to access them without reading the UML boot
log. It is also unique in allowing access to a UML from remote
machines without requiring that the UML be networked. This could be
useful in allowing public access to UMLs because they would be
accessible from the net, but wouldn't need any kind of network
filtering or access control because they would have no network
access.
If you attach the main console to a portal, then the UML boot will
appear to hang. In reality, it's waiting for a telnet to connect, at
which point the boot will proceed.
If you set up a file descriptor on the UML command line, you can
attach a UML device to it. This is most commonly used to put the main
console back on stdin and stdout after assigning all the other
consoles to something else:
con0=fd:0,fd:1 con=pts
Nothing - device=null
This allows the device to be opened, in contrast to 'none', but reads
will block, and writes will succeed and the data will be thrown out.
None - device=none
This causes the device to disappear. If you are using devfs, the
device will not appear in /dev. If not, then attempts to open it will
return -ENODEV.
You can also specify different input and output channels for a device
by putting a comma between them:
ssl3=tty:/dev/tty2,xterm
will cause serial line 3 to accept input on the host's /dev/tty3 and
display output on an xterm. That's a silly example - the most common
use of this syntax is to reattach the main console to stdin and stdout
as shown above.
If you decide to move the main console away from stdin/stdout, the
initial boot output will appear in the terminal that you're running
UML in. However, once the console driver has been officially
initialized, then the boot output will start appearing wherever you
specified that console 0 should be. That device will receive all
subsequent output.
Examples
There are a number of interesting things you can do with this
capability.
First, this is how you get rid of those bleeding console xterms by
attaching them to host ptys:
con=pty con0=fd:0,fd:1
This will make a UML console take over an unused host virtual console,
so that when you switch to it, you will see the UML login prompt
rather than the host login
prompt:
con1=tty:/dev/tty6
You can attach two virtual machines together with what amounts to a
serial line as follows:
Run one UML with a serial line attached to a pty -
ssl1=pty
Look at the boot log to see what pty it got (this example will assume
that it got /dev/ptyp1).
Boot the other UML with a serial line attached to the corresponding
tty -
ssl1=tty:/dev/ttyp1
Log in, make sure that it has no getty on that serial line, attach
a terminal program like minicom to it, and you should see the login
prompt of the other virtual machine.
There are a number of reasons to want to run a distribution's install
procedure inside UML:
The result is an authentic, upgradable installation, which you might
not get by using a more ad-hoc method.
You can do a dry run of the install without needing to devote hardware
to it. This can tell you what the installation will involve, and
booting the result will give you an idea of what the distribution
looks like once installed.
It's good exercise for the kernel. Installation procedures are
somewhat unusual and they are good at exposing bugs in the kernel.
A couple of installation procedures have been made to run under UML:
Slackware - This one was the easiest so far. Slackware has a
completely text-based procedure and it is also willing to take your
word on what devices it should use as installation sources and
targets. It doesn't poke at the system and do sanity checks on
devices like other, more user-friendly, installations do. A Slackware
installation kit is available
here .
It contains a Slackware boot disk image and a README which will walk
you through the install. You can see the README
here, and a log of
a typical installation
here.
SuSE - SuSE is pickier about the system it installs on. It looks at
/proc/partitions to figure out what devices are available as
installation targets, which requires the ubd block device to fake
partition information. It doesn't seem to have a textual install.
Instead, it uses ASCII graphics which only seem to display right on a
virtual console. I haven't found a terminal emulator which will do
the right thing with it. So, in order to run the SuSE installation,
you have to run the kernel on one of the host virtual consoles.
RedHat - It takes a little hacking to get RedHat to install cleanly
on a UML system, but Oleg Drokin has suceeded in getting RedHat up and
running. He has produced a document with lots of
screenshots if you want to do it yourself.
Debian - The Debian UML maintainer, Matt Zimmerman has managed to
get Debian Woody to install under UML with only a number of small issues.
You can read all about his progress at lists.debian.org where his original post to debian-boot can be
read.
Linux From Scratch - LFS is a slightly different kettle of fish, as we
have to compile our own Linux distribution, then boot it on UML. There
is a great HOW-TO
on using LFS with UML over at hints.linuxfromscratch.org.
Thanks to Greg Lonnon, UML (as of
2.4.6-4um) has support for I/O memory emulation. This allows a host
file to be specified as an I/O region on the kernel command line.
That file will be mapped into UML's kernel address space where a
driver can locate it and do whatever it wants with the memory,
including providing an interface to it for UML processes to use.
Driver interface
Specifying an iomem region on the kernel command line is done as
follows:
iomem=name,file
The name is defined by the driver that will use the I/O area and the
file is the name of the host file that will be mapped in.
The driver finds its I/O area by calling find_iomem
unsigned long find_iomem(char *driver, unsigned long *len_out)
which returns the address of the beginning of the area and passes its
length out in len_out. The driver argument is the name of the I/O
area as defined on the command line.
Once the driver has located its I/O area, it can do whatever it wants
with it, including
providing a file interface to it
allowing processes to mmap it into their own address spaces
use it as a backing store for some other kind of interface, such as an
X server using it as a framebuffer and providing an X protocol
interface to it
The mmapper driver
arch/um/drivers/mmapper_kern.c contains a simple example driver which
locates the iomem area named "mmapper" and makes it available for
mapping through /dev/mmapper.
Here is a little
program which will mmap the /dev/mmapper device and print its contents.
To run this driver, compile it into the kernel by turning on
CONFIG_MMAPPER, and assign a file to it by putting
UML is a fully-functional Linux kernel. It runs its own scheduler and
VM system, relying on the host kernel only for the equivalent of hardware
support.
The virtual machine is completely configurable through the command
line, allowing its memory, available devices, and their configurations
to be specified. The available command line switches are described
here.
Hardware support
UML hardware support comes in the form of virtual devices which use
resources on the host. It supports all the usual types of devices
that a physical system does:
Block devices - UML block devices are normally associated with a file
on the host which contains a filesystem. This file is made to look
like a block device inside the virtual machine, and the filesystem on
it is mounted just as a filesystem on a physical disk is mounted in a
native kernel. A block device file may also contain a swap signature,
in which case, UML will be willing to swap onto it. It can also be
treated as a raw disk, which isn't mounted, but instead has data read
from and written to it with a utility such as dd.
Block devices can also be attached to anything on the host which resembles
a file. Examples include CD-ROMS, floppies, physical disks and
partitions, and memory devices such as /dev/mem.
UML block devices can be layered, with a read-only device having a
copy-on-write (COW) read-write device on top of it. This acts as a
single read-write device, with the modifications to the read-only
layer being recorded in the read-write COW layer. This allows
multiple machines to share a filesystem, allowing a large saving in
disk space on the host. If you are doing development, and plan on
crashing UML frequently, then COW-ing your root filesystem allows you
to avoid fscking it on each boot by deleting the COW file and reusing
the clean backing file. This is described further
here.
Consoles and serial lines - These are almost identical, sharing most
of their code, and differing only where they need to present different
interfaces to the rest of the kernel. They can be attached to a
number of different interfaces on the host, including file
descriptors, ptys, ttys, pts devices, and xterms. More information
about this is here.
By default, the main console is attached to file descriptors 0 and 1
(stdin and stdout), the other consoles come up in xterms, and the
serial lines attach themselves to ptys.
Network devices - Similarly, the network driver can use a variety of
host interfaces to exchange packets with other virtual machines, the
host, and the rest of the network. To construct a completely virtual
network, with no access to the host networking, UML can use a hub
daemon to forward packets from one virtual machine to another or it
can set up a multicast network and have the machines send packets
directly to each other. To access the host and the outside network,
UML can use either ethertap or slip interfaces on the host. A more
complete description of this is
here.
SCSI devices - The SCSI subsystem compiles into UML cleanly and is
available as a config option. The only low-level driver currently
available for it is the scsi-debug driver, which operates in memory.
It grabs 8M at boot and runs a little SCSI ramdisk in it.
USB devices - James McMechan has written a UML hcd driver. I haven't
played with it yet, but it apparently allows UML to control physical
USB devices that the host hasn't claimed.
Sound - Thanks to the warped mind of Steve Schmidtke, UML has sound
support. The UML sound driver relays the UML's /dev/sound/dsp and
/dev/sound/mixer out to the host's /dev/sound/dsp and
/dev/sound/mixer. The UML needs appropriate permissions to access
these devices on the host.
PCI hardware - There are two efforts to allow UML two access physical
PCI hardware, allowing it to load and run the native drivers for those
devices. Jon Smirl has been working on a Virtual PCI patch, and has
been having some success with getting UML to access the host's frame
buffer. His latest announcement is
here .
Totally independently of this, Eitan Yarden
posted the abstract to his
thesis and the patch, which implements PCI hardware access from UML.
His effort is functional enough that he was able to drive an ethernet
card from the inside UML.
This isn't in UML yet, but will be merged at some point.
Hot-plugging devices
The management console is a
low-level interface into the kernel. Among other things, it allows
UML devices to be added to and removed from a running system.
Currently, the block devices and network devices can be hot-plugged.
In time, this will be extended to the rest of the devices, so you will
be able to construct, destroy, and rebuild essentially the entire
system without having to reboot it.
Generic kernel functionality
UML runs its scheduler independently of the host scheduler - the host
scheduler simply implements the decisions made by the UML scheduler.
Similarly, UML runs its own VM system, allocating backing memory from
a pool that it considers to be physical memory, mapping that memory as
needed into its process virtual address spaces and its own virtual
memory. If needed, and it has swap configured, it will swap.
Basically, anything in Linux that's not hardware-specific just works.
This includes kernel modules, the filesystems, software devices, and
network protocols.
Also, SMP and highmem are supported. They need to be configured in,
and once they are, they work the same as on any other architecture
that supports them.
UML/ppc - This is Chris Emerson's page on his port of UML to
PowerPC.
gbootroot
- A distribution-creation tool with support for creating UML
root filesystems.
BusyBox
- A Swiss Army knife Linux utility which combines a number of
utilities into one binary. The UML connection is that they use it to
test their stuff and distribute a UML that's been preconfigured for
BusyBox.
usermodelinux.co.uk -
Precompiled kernels and Linux From Scratch filesystems
The a386 Linux port
Lars
Brinkhoff has ported Linux to his
a386
virtual machine. The a386 is an abstract virtual machine which
resembles, but isn't identical to, a physical IA32 machine.
Plex86 and VMWare
These are virtual IA32 machines.
VMWare can
boot native Linux and Windows kernels.
Plex86 is at an
earlier stage of development, but it's intended to have the same
capabilities.
There are two Linux on S/390 projects,
IBM's
official port and the
skunkworks
port done by non-IBM people in Germany. UML is essentially the
same idea as IBM's porting its mainframe operating systems to its VM OS.
Minix on Solaris
This is a
port of
Minix to run on Solaris. It is exactly the same idea as UML,
except the guest OS is Minix rather than Linux and the host OS is
Solaris rather than Linux.
Hercules
This is an
S/390
emulator which runs on IA32 and S/390-compatible boxes.
Hercules emulates IBM S/370, S/390 and the new 64-bit zSeries well
enough to run their operating systems, including Linux/390 on IA32
CPUs.
OSKit
From a quick look at
this site
, it seems to be a userspace framework on top of which you can
write on OS. Same basic idea as UML, except more virtual.
Linux
User magazine ran an article on UML in their November issue.
The article is available
here as a pdf file. It's a pretty decent article, although
fairly lightweight technically.
I wrote an
article for O'Reilly on using UML to simulate disasters and practice
recovering from them. Linux Today
picked up on it. LWN didn't, but redeemed themselves by running an
item I sent them about the
UML Sysadmin Disaster of the
Month contest.
NewForge ran an article
about Linux virtual machines which completely failed to
mention UML. The guy who wrote the article was informed of this and
nicely made amends with this
article, which took the form of an interview.
There's also an
article on securitywriters.org about using UML as the basis of
a honeynet.
The UML management console is a low-level interface to the kernel,
somewhat like the i386 SysRq interface. Since there is a full-blown
operating system under UML, there is much greater flexibility possible
than with the SysRq mechanism.
There are a number of things you can do with the mconsole interface:
get the kernel version
add and remove devices
halt or reboot the machine
send SysRq commands
pause and resume the UML
make online backups without shutting down the UML
receive notifications of events of interest from within UML
monitor the internal state of the UML
You need the mconsole client (uml_mconsole) which is present in CVS
(/tools/mconsole) in 2.4.5-9um and later, and will be in the RPM in 2.4.6.
You also need CONFIG_MCONSOLE (under 'General Setup') enabled in UML.
When you boot UML, you'll see a line like:
mconsole initialized on /home/jdike/.uml/umlNJ32yL/mconsole
If you specify a unique machine id one the UML command line, i.e.
umid=debian
you'll see this
mconsole initialized on /home/jdike/.uml/debian/mconsole
That file is the socket that uml_mconsole will use to communicate
with UML. Run it with either the umid or the full path as its argument:
You'll get a prompt, at which you can run one of these commands:
version
halt
reboot
config
remove
sysrq
help
cad
stop
go
log
proc
version
This takes no arguments. It prints the UML version.
(mconsole) version
OK Linux usermode 2.4.5-9um #1 Wed Jun 20 22:47:08 EDT 2001 i686
There are a couple actual uses for this. It's a simple no-op which
can be used to check that a UML is running. It's also a way of
sending an interrupt to the UML. This is sometimes useful on SMP
hosts, where there's a bug which causes signals to UML to be lost,
often causing it to appear to hang. Sending such a UML the mconsole
version command is a good way to 'wake it up' before networking has
been enabled, as it does not do anything to the function of the UML.
halt and reboot
These take no arguments. They shut the machine down immediately, with
no syncing of disks and no clean shutdown of userspace. So, they are
pretty close to crashing the machine.
(mconsole) halt
OK
config
"config" adds a new device to the virtual machine or queries the configuration of an existing device.
Currently the ubd and network drivers support pulling devices. It takes one
argument, which is the device to add, with the same syntax as the
kernel command line.
(mconsole)
config ubd3=/home/jdike/incoming/roots/root_fs_debian22
OK
(mconsole) config eth1=mcast
OK
Querying the configuration of a device is handy when you don't know
before the boot what host device the UML device will attach to. This
is a problem with attaching consoles and serial lines to host pty or
pts devices. You have no way of knowing how to access them without
parsing the kernel messages. So, the syntax for this is the same as
above, except you don't specify a configuration
(mconsole) config ssl0
OK pty:/dev/ptyp0
(mconsole) config ubd0
OK /home/jdike/roots/cow.debian,/home/jdike/roots/debian_22
This is supported by the console, serial line, and ubd drivers. As
yet, the network drivers don't support this.
remove
"remove" deletes a device from the system. Its argument is just the
name of the device to be removed. The device must be idle in
whatever sense the driver considers necessary. In the case of the ubd
driver, the removed block device must not be mounted, swapped on, or
otherwise open, and in the case of the network driver, the device must be down.
(mconsole) remove ubd3
OK
(mconsole) remove eth1
OK
sysrq
This takes one argument, which is a single letter. It calls the
generic kernel's SysRq driver, which does whatever is called for by
that argument. See the SysRq documentation in Documentation/sysrq.txt
in your favorite kernel tree to see what letters are valid and what
they do.
help
"help" returns a string listing the valid commands and what each one does.
cad
This invokes the Ctl-Alt-Del action on init. What exactly this ends
up doing is up to /etc/inittab. Normally, it reboots the machine.
With UML, this is usually not desired, so if a halt would be better,
then find the section of inittab that looks like this
# What to do when CTRL-ALT-DEL is pressed.
ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
and change the command to halt.
stop
This puts the UML in a loop reading mconsole requests until a 'go'
mconsole command is recieved. This is very useful for making backups of
UML filesystems, as the UML can be stopped, then synced via 'sysrq s', so
that everything is written to the filesystem. You can then copy the
filesystem and then send the UML 'go' via mconsole.
Note that a UML running with more than one CPU will have problems after
you send the 'stop' command, as only one CPU will be held in a mconsole
loop and all others will continue as normal. This is a bug, and will
be fixed.
go
This resumes a UML after being paused by a 'stop' command. Note that
when the UML has resumed, TCP connections may have timed out and if the
UML is paused for a long period of time, crond might go a little crazy,
running all the jobs it didn't do earlier.
log
This takes a string as its argument, and will cause the UML to printk
the string so that it ends up in the kernel message log. This is
intended for use in honeypots by allowing the UML-specific stuff in
the kernel log to be replaced with messages that don't expose the
machine as being a UML.
proc
This takes a filename as its argument. It will return the contents of
the corresponding /proc file inside the UML. Example:
(mconsole) proc uptime
will return the contents of the UML's /proc/uptime.
Making online backups
It is possible to make a backup of a UML's data without shutting it
down. The idea is to pause it, make it flush out its data, copy the
filesystem to a safe place, and then resume it. This should usually
take seconds, while shutting down and rebooting the UML could take
minutes. The exact procedure is this:
(mconsole) stop
(mconsole) sysrq s
host% # Copy the UML's filesystem someplace safe
(mconsole) go
By causing UML to flush its data out to disk, the 'sysrq s' will cause
the filesystem to be a clean image. Of course, no guarantees are made
for process data which hadn't been written back to the kernel, but the
filesystem itself won't need an fsck if it's booted.
Event notification
The mconsole interface also provides a mechanism for processes inside
a UML to send messages to an mconsole client on the host. The
procedure is this:
Create a unix socket and pass that to UML on the command line as the
mconsole notification socket
mconsole=notify:socket
A /proc/mconsole file will be created inside UML
Anything that is written to it will be turned into an mconsole
notification which your mconsole client should be listening for on the
notification socket
A common use for this mechanism is to have an rc script inside UML
send a message out that the UML has booted to a certain stage, and
that something on the host which depends on that can proceed.
However, this is a completely general mechanism which can be used to
communicate any information at all to the host.
There is a demo mconsole notification client in the utilities tarball
in mconsole/notify.pl. This is only a demo, and as such, isn't very
useful by itself. It should be customized to fit into whatever
environment you are setting up.
You can't run a UML binary, copy that same executable in, run it and
expect it to work. The inner one will try to load on top of the outer
one.
So, with UML 2.4.18-23 and earlier, you need to build the inner
UML with 'NESTING=1' on the make line:
host% make linux ARCH=um NESTING=1
With later versions, set CONFIG_NEST_LEVEL (under 'General Setup') to 1.
See this page for
information on building UML if you are not familiar with this
process. Note that the outer UML must be at least 2.4.18-22, which is
not released at this writing, but will be soon.
Running the nested UML
Boot the normal UML and copy the 'NESTING=1' into it. You also need
to provide a root filesystem. You can copy one in, but it's easier
just to make it available as a block device. Either
ubd1=inner_root_fs
on the command line or
host%
uml_mconsole umid config ubd1=inner_root_fs
after booting will make the root_fs on the host available as
/dev/ubd/1 inside UML. You can use the same root filesystem as the
outer UML if both UMLs COW it.
Boot the inner UML as
UML#
linux ubd1=/dev/ubd/1
plus whatever switches you want.
The first nested UML boot log
This is the log of the first nested UML boot.
'usermode:~#' is the UML shell prompt.
/dev/ubd/1 has been attached to a tomsrtbt image on the host:
~/linux/2.4/nest/linux is the specially built nested UML.
Also note the back-to-back UML shutdowns at the very end.
usermode:~# scp jdike@192.168.0.254:linux/2.4/nest/linux .
jdike@192.168.0.254's password:
scp: warning: Executing scp1 compatibility.
linux 100% |*****************************| 8227 KB 00:00 ETA
usermode:~# ls -l linux
-rwxr--r-- 1 root root 8424800 May 3 21:09 linux
usermode:~# ./linux ubd0=/dev/ubd/1
tracing thread pid = 146
Linux version 2.4.18-21um (jdike@uml.karaya.com) (gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-81)) #1 Wed May 1 21:07:32 EDT 2002
On node 0 totalpages: 8192
zone(0): 0 pages.
zone(1): 8192 pages.
zone(2): 0 pages.
Kernel command line: ubd0=/dev/ubd/1 root=/dev/ubd0
Calibrating delay loop... 707.26 BogoMIPS
Memory: 32244k available
Dentry-cache hash table entries: 4096 (order: 3, 32768 bytes)
Inode-cache hash table entries: 2048 (order: 2, 16384 bytes)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Buffer-cache hash table entries: 1024 (order: 0, 4096 bytes)
Page-cache hash table entries: 8192 (order: 3, 32768 bytes)
Checking for host processor cmov support...Reading /proc/cpuinfo failed, errno = 2
Yes
Checking for host processor xmm support...Reading /proc/cpuinfo failed, errno = 2
No
Checking that ptrace can change system call numbers...OK
Checking that host ptys support output SIGIO...No, enabling workaround
POSIX conformance testing by UNIFIX
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
Starting kswapd
VFS: Diskquotas version dquot_6.4.0 initialized
devfs: v1.10 (20020120) Richard Gooch (rgooch@atnf.csiro.au)
devfs: boot_options: 0x1
JFFS version 1.0, (C) 1999, 2000 Axis Communications AB
JFFS2 version 2.1. (C) 2001 Red Hat, Inc., designed by Axis Communications AB.
pty: 256 Unix98 ptys configured
block: 64 slots per queue, batch=16
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
loop: loaded (max 8 devices)
PPP generic driver version 2.4.1
Universal TUN/TAP device driver 1.4 (C)1999-2001 Maxim Krasnyansky
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 2048 bind 2048)
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
Initializing stdio console driver
Initializing software serial port version 1
mconsole (version 1) initialized on /root/.uml/yfWc2E/mconsole
Partition check:
ubda: unknown partition table
UML Audio Relay: May 1 2002 21:17:10
VFS: Mounted root (ext2 filesystem) readonly.
Mounted devfs on /dev
INIT: version 2.84 booting
EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended
INIT: Entering runlevel: 5
Welcome to the uml version of Tom's root/boot.
This is a customized version, so the notice below is displayed
at Tom's request.
*******************************************************************************
* If you base something on it, use any of the scripts, distribute binaries or *
* libraries from it, or distribute customized versions of it: You must credit *
* tomsrtbt and include a pointer to http://www.toms.net/rb/ and tom@toms.net, *
* and include this notice verbatim. Copyright Tom Oehser 1999. This notice in *
* no way supercedes or nullifies any other protections on the component parts *
* such as the BSD and GPL copyrights which apply to practically everything!!! *
* Within these strictures you may redistribute, incorporate, copy, modify, or *
* do anything else to it or with it that you like. Tomsrtbt has no warranties *
* not even implied fitness or usefulness. If it breaks you keep both pieces. *
*******************************************************************************
What you have is...
3c589_cs advansys agetty aha152x aha152x_cs aha1542 aic7xxx ash awk badblocks
bdflush buildit.s busLogic busybox bzip2 cardbus cardmgr cat ce ce.help chain.b
chattr chgrp chmod chown chroot clear clone.s cmp common config cp cpio cut date
dd ddate debugfs df dhcpcd dirname dmesg ds du dumpe2fs e2fsck eata echo echo.c
elvis emacs ex extend false fdflush fdformat fdisk fdomain filesize find
findsuper fmt fsck.ext2 fsck.msdos fstab grep group gzip halt head hexdump
hexedit host.conf hostname hosts i82365 ifconfig ifport ile image init init.old
inittab insmod install.s ioctl.save issue kill killall5 ksyms ld ld-linux length
less libc libcom_err libe2p libext2fs libss libtermcap libuuid lilo lilo.conf ln
loadkeys login losetup ls lsattr man mawk md5sum memtest miterm mkdir mkdosfs
mke2fs mkfifo mkfs.minix mklost+found mknod mkswap mnsed more more.help mount mt
mtab mv nc ncr53c8xx network networks nmclan_cs ntfs passwd pax pcmcia
pcmcia_core pcnet_cs ping plip ppa printf profile protocols ps pwd qlogic_cs
qlogicfas rc.0 rc.6 rc.M rc.S rc.custom rc.custom.gz rc.custom~ reboot rescuept
reset resolv.conf rm rmdir rmmod route rsh rshd script scsi scsi_info seagate
sed serial serial_cs services setserial settings.s sh shared shutdown slattach
sleep snarf sort split stty swapoff swapon sync tail tar tcic tee telnet telnetd
termcap test tomcr.txt tomshexd tomsrtbt.FAQ touch true tune2fs umount undeb
undeb-- unpack.s unrpm-- update utmp vi vi.help view wc wtmplock
...Login as root.
ttys/0 tomsrtbt login: root
Password: ile rev.2.01
Today is Pungenday, the 50th day of Discord in the YOLD 3168
Celebrate Discoflux
# ps uax
1 S (init) init
2 S (keventd)
3 S (ksoftirqd_CPU0)
4 S (kswapd)
5 S (bdflush)
6 S (kupdated)
7 S (mtdblockd)
55 S (ile) ile /bin/sh -c . /etc/profile -si
59 S (sh) /bin/sh -c . /etc/profile -si
61 S (ps) /bin/sh /usr/bin/ps uax
62 R (ps) /bin/sh /usr/bin/ps uax
63 R (sed) sed -e s/\ / /g
# halt
INIT: Switching to runlevel: 0
INIT: Sending processes the TERM signal
halt
System halted.
nbd: module cleaned up.
usermode:~# halt
Broadcast message from root (vc/0) Fri May 3 21:24:09 2002...
The system is going down for system halt NOW !!
INIT: Switching to runlevel: 0
INIT: Sending processes the TERM signal
INIT: Sending processes the KILL signal
Stopping internet superserver: inetd.
Stopping OpenBSD Secure Shell server: sshd.
Saving the System Clock time to the Hardware Clock...
hwclock: Can't open /dev/tty1, errno=2: No such file or directory.
hwclock is unable to get I/O port access: the iopl(3) call failed.
Hardware Clock updated to Fri May 3 21:25:17 CEST 2002.
Stopping portmap daemon: portmap.
Stopping NFS kernel daemon: mountd nfsd.
Unexporting directories for NFS kernel daemon...done.
Stopping NFS common utilities: statd.
Stopping system log daemon: klogd syslogd.
Sending all processes the TERM signal... done.
Sending all processes the KILL signal... done.
Saving random seed... done.
Unmounting remote filesystems... done.
Deconfiguring network interfaces: done.
Deactivating swap... done.
Unmounting local filesystems... done.
* route del -host 192.168.0.253 dev tap0
* bash -c echo 0 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
* arp -i eth0 -d 192.168.0.253 pub
Power down.
nbd: module cleaned up.
um 1012:
Hosted at
user-mode-linux-doc-20060501/net.html 0000644 0000000 0000000 00000025673 12742461304 014133 0 ustar
A virtual network and X session
The screenshot below shows two virtual machines, one of which is
running X. Click on it for a full-size version.
The four xterms on the left are the Slackware virtual machine's consoles,
and the four on the right are the SuSE virtual machine's three
consoles, plus its syslog in a fourth console. The Xnest in back is
the local X server for the Slackware machine. The xterm inside it
running on the SuSE machine, displaying over to the Slackware X
server. The shell inside the xterm is telnetted back to the Slackware
machine.
Hosted at
user-mode-linux-doc-20060501/networking.html 0000644 0000000 0000000 00000135550 12742461304 015530 0 ustar
Setting up the network
This page describes how to set up the various transports and to
provide a UML instance with network access to the host, other machines
on the local net, and the rest of the net.
As of 2.4.5, UML networking has been completely redone to make it much
easier to set up, fix bugs, and add new features.
There is a new helper, uml_net, which does the host setup that
requires root privileges.
There are currently five transport types available for a UML virtual
machine to exchange packets with other hosts:
ethertap
TUN/TAP
Multicast
a switch daemon
slip
slirp
pcap
The TUN/TAP, ethertap, slip, and slirp transports allow a UML instance to
exchange packets with the host. They may be directed to the host or
the host may just act as a router to provide access to other physical
or virtual machines.
The pcap transport is a synthetic read-only interface, using the libpcap
binary to collect packets from interfaces on the host and filter them.
This is useful for building preconfigured traffic monitors or sniffers.
The daemon and multicast transports provide a completely virtual
network to other virtual machines. This network is completely
disconnected from the physical network unless one of the virtual
machines on it is acting as a gateway.
With so many host transports, which one should you use? Here's when
you should use each one:
ethertap - if you want access to the host networking and it is running
2.2
TUN/TAP - if you want access to the host networking and it is running
2.4. Also, the TUN/TAP transport is able to use a preconfigured
device, allowing it to avoid using the setuid uml_net helper, which
is a security advantage.
Multicast - if you want a purely virtual network and you don't want to
set up anything but the UML
a switch daemon - if you want a purely virtual network and you don't
mind running the daemon in order to get somewhat better performance
slip - there is no particular reason to run the slip backend unless
ethertap and TUN/TAP are just not available for some reason
slirp - if you don't have root access on the host to setup networking,
or if you don't want to allocate an IP to your UML
pcap - not much use for actual network connectivity, but great for
monitoring traffic on the host
Ethertap is available on 2.4 and works fine. TUN/TAP is preferred to
it because it has better performance and ethertap is officially
considered obsolete in 2.4. Also, the root helper only needs to run
occasionally for TUN/TAP, rather than handling every packet, as it
does with ethertap. This is a slight security advantage since it
provides fewer opportunities for a nasty UML user to somehow exploit
the helper's root privileges.
General setup
First, you must have the virtual network enabled in your UML. If are
running a prebuilt kernel from this site, everything is already enabled.
If you build the kernel yourself, under the "Network device support" menu,
enable "Network device support", and then the three transports.
The next step is to provide a network device to the virtual machine.
This is done by describing it on the kernel command line.
The general format is
eth<n>=<transport>,<transport args>
For example, a virtual ethernet device may be attached to a host ethertap
device as follows:
eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
This sets up eth0 inside the virtual machine to attach itself to the
host /dev/tap0, assigns it an ethernet address, and assigns the host
tap0 interface an IP address.
Note that the IP address you assign to the host end of the tap device
must be different than the IP you assign to the eth device inside
UML. If you are short on IPs and don't want to comsume two per
UML, then you can reuse the host's eth IP address for the host ends of
the tap devices. Internally, the UMLs must still get unique IPs for
their eth devices. You can also give the UMLs non-routable IPs
(192.168.x.x or 10.x.x.x) and have the host masquerade them. This
will let outgoing connections work, but incoming connections won't
without more work, such as port forwarding from the host.
Also note that when you configure the host side of an interface, it is only
acting as a gateway. It will respond to pings sent to it locally, but
is not useful to do that since it's a host interface. You are not
talking to the UML when you ping that interface and get a response.
You can also add devices to a UML and remove them at runtime. See
the mconsole page for details.
The sections below describe this in more detail.
Once you've decided how you're going to set up the devices, you boot
UML, log in, configure the UML side of the devices, and set up routes
to the outside world. At that point, you will be able to talk to any
other machines, physical or virtual, on the net.
If ifconfig inside UML fails and the network refuses to come up, run
'dmesg' to see what ended up in the kernel log. That will usually
tell you what went wrong.
Userspace daemons
You will likely need the setuid helper, or the switch daemon, or
both. They are both installed with the RPM and deb, so if you've installed
either, you can skip the rest of this section.
If not, then you need to check them out of CVS ,
build them, and install them. The helper is uml_net, in CVS
/tools/uml_net, and the daemon is uml_switch, in CVS
/tools/uml_router. They are both built with a plain 'make'. Both
need to be installed in a directory that's in your path - /usr/bin is
recommend. On top of that, uml_net needs to be setuid root.
Specifying ethernet addresses
Below, you will see that the TUN/TAP, ethertap, and daemon interfaces
allow you to specify hardware addresses for the virtual ethernet
devices. This is generally not necessary. If you don't have a
specific reason to do it, you probably shouldn't. If one is not
specified on the command line, the driver will assign one based on the
device IP address. It will provide the address fe:fd:nn:nn:nn:nn
where nn.nn.nn.nn is the device IP address. This is nearly always
sufficient to guarantee a unique hardware address for the device. A
couple of exceptions are:
Another set of virtual ethernet devices are on the same network and
they are assigned hardware addresses using a different scheme which
may conflict with the UML IP address-based scheme
You aren't going to use the device for IP networking, so you don't
assign the device an IP address
If you let the driver provide the hardware address, you should make sure
that the device IP address is known before the interface is brought
up. So, inside UML, this will guarantee that:
UML#
ifconfig eth0 192.168.0.250 up
If you decide to assign the hardware address yourself, make sure that
the first byte of the address is even. Addresses with an odd first byte
are broadcast addresses, which you don't want assigned to a device.
UML interface setup
Once the network devices have been described on the command line, you
should boot UML and log in.
The first thing to do is bring the interface up:
UML# ifconfig ethnip-address up
You should be able to ping the host at this point.
To reach the rest of the world, you should set a default route to the
host:
UML# route add default gw host ip
Again, with host ip of 192.168.0.4:
UML# route add default gw 192.168.0.4
This page used to recommend setting a network route to your local
net. This is wrong, because it will cause UML to try to figure out
hardware addresses of the local machines by arping on the interface to
the host. Since that interface is basically a single strand of
ethernet with two nodes on it (UML and the host) and arp requests
don't cross networks, they will fail to elicit any responses. So,
what you want is for UML to just blindly throw all packets at the host
and let it figure out what to do with them, which is what leaving out
the network route and adding the default route does.
Note: If you can't communicate with other hosts on your physical
ethernet, it's probably because of a network route that's
automatically set up. If you run 'route -n' and see a route that
looks like this:
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
with a mask that's not 255.255.255.255, then replace it with a route
to your host:
UML#
route del -net 192.168.0.0 dev eth0 netmask 255.255.255.0
UML#
route add -host 192.168.0.4 dev eth0
This, plus the default route to the host, will allow UML to exchange
packets with any machine on your ethernet.
Multicast
The simplest way to set up a virtual network between multiple UMLs is
to use the mcast transport. This was written by Harald Welte and is
present in UML version 2.4.5-5um and later. Your system must have
multicast enabled in the kernel and there must be a multicast-capable
network device on the host. Normally, this is eth0, but if there is
no ethernet card on the host, then you will likely get strange error
messages when you bring the device up inside UML.
To use it, run two UMLs with
eth0=mcast
on their command lines. Log in, configure the ethernet device in each
machine with different IP addresses:
UML1# ifconfig eth0 192.168.0.254
UML2# ifconfig eth0 192.168.0.253
and they should be able to talk to each other.
The full set of command line options for this transport are
Harald's original README is
here and explains
these in detail, as well as some other issues.
TUN/TAP with the uml_net helper
TUN/TAP is the preferred mechanism on 2.4 to exchange packets with the
host. The TUN/TAP backend has been in UML since 2.4.9-3um.
The easiest way to get up and running is to let the setuid uml_net
helper do the host setup for you. This involves insmod-ing the tun.o
module if necessary, configuring the device, and setting up IP
forwarding, routing, and proxy arp. If you are new to UML networking,
do this first. If you're concerned about the security implications of
the setuid helper, use it to get up and running, then read the next
section to see how to have UML use a preconfigured tap device, which
avoids the use of uml_net.
If you specify an IP address for the host side of the device, the
uml_net helper will do all necessary setup on the host - the only
requirement is that TUN/TAP be available, either built in to the host
kernel or as the tun.o module.
The format of the command line switch to attach a device to a TUN/TAP device
is
eth<n>=tuntap,,,<host IP address>
For example, this argument will attach the UML's eth0 to
the next available tap device, assign the IP address
192.168.0.254 to the host side of the tap device, and
assign an ethernet address to it based on the IP address assigned to
it by ifconfig inside UML.
eth0=tuntap,,,192.168.0.254
If you using the uml_net helper to set up the host side of the
networking, as in this example, note that changing the UML IP address
will cause uml_net to change the host routing and arping to match.
This is one reason you should not be using uml_net if there is any
possibility that the user inside the UML may be unfriendly. This
feature is convenient, but can be used to make the UML pretend to be
something like your name server or mail server, and the host will
steal packets intended for those servers and forward them to the UML.
See the next section for setting up networking in a secure manner.
There are a couple potential problems with running the TUN/TAP
transport on a 2.4 host kernel
TUN/TAP seems not to work on 2.4.3 and earlier. Upgrade the host
kernel or use the ethertap transport.
With an upgraded kernel, TUN/TAP may fail with
File descriptor in bad state
This is due to a header mismatch between the upgraded kernel and the
kernel that was originally installed on the machine. The fix is to
make sure that /usr/src/linux points to the headers for the running kernel.
If you prefer not to have UML use uml_net (which is somewhat
insecure), with UML 2.4.17-11, you can set up a TUN/TAP device
beforehand. The setup needs to be done as root, but once that's done,
there is no need for root assistance. Setting up the device is done
as follows:
Create the device with tunctl (available from the UML utilities
tarball)
host# tunctl -u uid
where uid is the user id or username that UML will be run as.
This will tell you what device was created.
Configure the device IP (change IP addresses and device name to suit)
host# ifconfig tap0 192.168.0.254 up
Set up routing and arping if desired - this is my recipe, there are
other ways of doing the same thing
Note that this must be done every time the host boots - this
configuration is not stored across host reboots. So, it's probably a
good idea to stick it in an rc file. An even better idea would be a
little utility which reads the information from a config file and sets
up devices at boot time.
Rather than using up two IPs and ARPing for one of them, you can also
provide direct access to your LAN by the UML by using a bridge.
host#
brctl addbr br0
host#
ifconfig eth0 0.0.0.0 promisc up
host#
ifconfig tap0 0.0.0.0 promisc up
host#
ifconfig br0 192.168.0.1 netmask 255.255.255.0 up
host#
brctl stp br0 off
host#
brctl setfd br0 1
host#
brctl sethello br0 1
host#
brctl addif br0 eth0
host#
brctl addif br0 tap0
Note that 'br0' should be setup using ifconfig with the existing IP
address of eth0, as eth0 no longer has its own IP.
Also, the /dev/net/tun device must be writable by the user running UML
in order for the UML to use the device that's been configured for it.
The simplest thing to do is
host# chmod 666 /dev/net/tun
Making it world-writeable looks bad, but it seems not to be
exploitable as a security hole. However, it does allow anyone to
create useless tap devices (useless because they can't configure
them), which is a DOS attack. A somewhat more secure alternative
would to be to create a group containing all the users who have
preconfigured tap devices and chgrp /dev/net/tun to that group with
mode 664 or 660.
Once the device is set up, run UML with
eth0=tuntap,devicename
i.e.
eth0=tuntap,tap0
on the command line (or do it with
the mconsole config command).
Bring the eth device up in UML and you're in business.
If you don't want that tap device any more, you can make it
non-persistent with
host# tunctl -d tap device
Finally, tunctl has a -b (for brief mode) switch which causes it to
output only the name of the tap device it created. This makes it
suitable for capture by a script:
host# TAP=`tunctl -u 1000 -b`
Ethertap
Ethertap is the general mechanism on 2.2 for userspace processes to
exchange packets with the kernel.
To use this transport, you need to describe the virtual network
device on the UML command line. The general format for this is
eth<n>=ethertap,<device>,<ethernet address>,<host IP address>
So, the previous example
eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
attaches the UML eth0 device to the host
/dev/tap0, assigns it the
ethernet address fe:fd:0:0:0:1, and assigns the IP address
192.168.0.254 to the host side of the tap device.
The tap device is mandatory, but the others are optional. If the
ethernet address is omitted, one will be assigned to it.
The presence of the tap IP address will cause the helper to run and do
whatever host setup is needed to allow the virtual machine to
communicate with the outside world. If you're not sure you know what
you're doing, this is the way to go.
If it is absent, then you must configure the tap device and whatever
arping and routing you will need on the host. However, even in this
case, the uml_net helper still needs to be in your path and it must be
setuid root if you're not running UML as root. This is because the
tap device doesn't support SIGIO, which UML needs in order to use
something as a source of input. So, the helper is used as a
convenient asynchronous IO thread.
If you're using the uml_net helper, you can ignore the following host
setup - uml_net will do it for you. You just need to make sure you
have ethertap available, either built in to the host kernel or
available as a module.
If you want to set things up yourself, you need to make
sure that the appropriate /dev entry exists. If it doesn't, become
root and create it as follows (the $[ ... ] is bash syntax for adding 16 to
the minor number) :
mknod /dev/tap<minor> c 36 $[ <minor> + 16 ]
For example, this is how to create /dev/tap0:
mknod /dev/tap0 c 36 $[ 0 + 16 ]
You also need to make sure that the host kernel has ethertap support.
If ethertap is enabled as a module, you apparently need to insmod
ethertap once for each ethertap device you want to enable. So,
host#
insmod ethertap
will give you the tap0 interface. To get the tap1 interface, you need
to run
host#
insmod ethertap unit=1 -o ethertap1
The switch daemon
Note: This is the daemon formerly known as uml_router, but
which was renamed so the network weenies of the world would stop
growling at me.
The switch daemon, uml_switch, provides a mechanism for creating a
totally virtual network. By default, it provides no connection to the
host network (but see -tap, below).
The first thing you need to do is run the daemon. Running it with no
arguments will make it listen on a default unix domain socket.
If you want it to listen on a different socket, use
-unix socket
If you want it to act as a hub rather than a switch, use
-hub
If you're planning on putting it in hub mode so you can sniff UML
traffic from a tap device on the host, it appears that you need to
assign the tap an IP address before you'll see any packets on it.
If you want the switch to be connected to host networking (allowing the
umls to get access to the outside world through the host), use
-tap tap0
Note that the tap device must be preconfigured (see "TUN/TAP with a
preconfigured tap device", above). If you're using a different tap
device than tap0, specify that instead of tap0.
The reason it doesn't background by default is that it listens to
stdin for EOF. When it sees that, it exits.
The general format of the kernel command line switch is
ethn=daemon,ethernet address,socket type,socket
You can leave off everything except the 'daemon'. You only need to
specify the ethernet address if the one that will be assigned to it
isn't acceptable for some reason. The rest of the arguments describe
how to communicate with the daemon. You should only specify them if
you told the daemon to use different sockets than the default. So, if
you ran the daemon with no arguments, running the UML on the same
machine with
eth0=daemon
will cause the eth0 driver to attach itself to the daemon correctly.
The socket argument is the filename of a Unix domain socket which is
used for communications between uml_switch and the UMLs on its network.
If you do specify a different socket from the default, which you will
need to do if you want multiple, separate uml_switch networks on the
host, you need to make sure that you name the same path for the socket
on both the uml_switch and UML command lines.
Currently the only supported value for the socket type is "unix".
Slip
Slip is another, less general, mechanism for a process to communicate
with the host networking. In contrast to the ethertap interface,
which exchanges ethernet frames with the host and can be used to
transport any higher-level protocol, it can only be used to transport
IP.
The general format of the command line switch is
ethn=slip,slip IP
The slip IP argument is the IP address that will be assigned to the
host end of the slip device. If it is specified, the helper will run
and will set up the host so that the virtual machine can reach it and
the rest of the network.
There are some oddities with this interface that you should be aware
of. You should only specify one slip device on a given virtual
machine, and its name inside UML will be 'umn', not 'eth0' or whatever
you specified on the command line. These problems will be fixed at some point.
Slirp
slirp uses an external program, usually /usr/bin/slirp, to provide IP
only networking connectivity through the host. This is similar to IP
masquerading with a firewall, although the translation is performed in
user-space, rather than by the kernel. As slirp does not set up any
interfaces on the host, or changes routing, slirp does not require
root access or setuid binaries on the host.
The general format of the command line switch for slirp is:
ethn=slirp,ethernet address,slirp path
The ethernet address is optional, as UML will set up the interface
with an ethernet address based upon the initial IP address of the
interface. The slirp path is generally /usr/bin/slirp, although it
will depend on distribution.
The slirp program can have a number of options passed to the command line
and we can't add them to the UML command line, as they will be parsed
incorrectly. Instead, a wrapper shell script can be written or the options
inserted into the ~/.slirprc file. More information on all of the slirp
options can be found in its man pages.
The eth0 interface on UML should be set up with the IP 10.2.0.15,
although you can use anything as long as it is not used by a network you
will be connecting to. The default route on UML should be set to use
'eth0' without a gateway IP:
UML#
route add default dev eth0
slirp provides a number of useful IP addresses which can be used by
UML, such as 10.0.2.3 which is an alias for the DNS server specified in
/etc/resolv.conf on the host or the IP given in the 'dns' option for slirp.
Even with a baudrate setting higher than 115200, the slirp connection
is limited to 115200. If you need it to go faster, the slirp binary
needs to be compiled with FULL_BOLT defined in config.h.
pcap
The pcap transport is attached to a UML ethernet device on the command
line or with uml_mconsole with the following syntax:
The interface is whatever network device on the host you want to sniff.
The expression is a pcap filter expression, which is also what tcpdump
uses, so if you know how to specify tcpdump filters, you will use the
same expressions here. The options are up to two of 'promisc',
'nopromisc', 'optimize', 'nooptimize'. 'promisc' and 'nopromisc'
control whether pcap puts the host interface into promiscuous
mode. 'optimize' and 'nooptimize' control whether the pcap expression
optimizer is used.
Example:
eth0=pcap,eth0,tcp
eth1=pcap,eth0,!tcp
will cause the UML eth0 to emit all tcp packets on the host eth0 and the
UML eth1 to emit all non-tcp packets on the host eth0.
Setting up the host yourself
If you don't specify an address for the host side of the ethertap or
slip device, UML won't do any setup on the host. So this is what is
needed to get things working (the examples use a host-side IP of
192.168.0.251 and a UML-side IP of 192.168.0.250 - adjust to suit your
own network):
The device needs to be configured with its IP address. Tap devices
are also configured with an mtu of 1484. Slip devices are configured
with a point-to-point address pointing at the UML ip address.
host# ifconfig tap0 arp mtu 1484 192.168.0.251 up
host#
ifconfig sl0 192.168.0.251 pointopoint 192.168.0.250 up
If a tap device is being set up, a route is set to the UML IP.
The purpose of this page is to keep people better informed about
ongoing work between UML releases by making the patches currently in
my working pool visible to the public. This should alleviate several
issues with UML development:
Not infrequently, someone finds a bug in UML, chases it down, and
submits a patch, not knowing that the bug has already been fixed in my
tree. Since my working tree isn't public until a release, there was
no way for anyone to know that the bug was already fixed.
Also not infrequently, the fixes in my tree are incomplete or wrong in
some other way. Having those patches available before the release
gives UML developers a way to test and sanity-check the patches before
they are released to the public.
Having the patches in a release split out makes it easier to fix new
bugs by allowing users to back out patches until the bug disappears.
Then we know which patch was responsible and can probably figure out
the problem quickly. This also allows non-expert users to help track
things down since the only expertise needed is the ability to run
patch and build UML from source.
To this end, I've started using quilt to
manage patches, and will publish the unreleased patches in my current
tree here. It will be updated frequently so that there will only be a
short window between me putting a patch in my tree and it appearing
here.
So, here are the patches pending in my 2.4 and 2.6 trees. The version
is the last public release of UML to which these are applied. They apply in
order.
From Gerd Knorr - this avoids a network deadlock that can happen when the
host side of an interface is full when the UML interface is brought up.
In this case, SIGIOs will never be delivered since no new data is ever
queued to the host side.
This is Blaisorblade's sysemu patch for UML, cleaned up some, and with
support added for tt mode. This adds support to UML for Laurent Vivier's
context-switch-reducing sysemu patch.
This fixes a use-after-free bug in the context switching. A process going
out of context after exiting wakes up the next process and then kills
itself. The problem is that when it gets around to killing itself is up to
the host and can happen a long time later, including after the incoming
process has freed its stack, and that memory is possibly being used for
something else.
The fix is to have the incoming process kill the exiting process just to
make sure it can't be running at the point that its stack is freed.
This forces make to use bash rather than whatever /bin/sh is linked to.
There are some bash extensions used in the build (and maybe this needs
fixing) and when /bin/sh isn't bash, then the build fails mysteriously.
Do not flush the whole kernel page table instead of flushing only a range
in SKAS mode:
loop from start to end instead than to start_vm to end_vm. To test a lot,
since it could well be wrong, or some callers could be passing wrong
parameters (they were ignored!). Anyway, it seems that this is safe and
that most callers are in arch-independent code (i.e. correct one).
But actually I did not test modules well.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade_spam@yahoo.it>
I eyeballed all the callers, and they seem to be doing the right
thing - jdike
This adds a check that /tmp is not mounted noexec. UML needs to be able
to do PROT_EXEC mmaps of temp files. Previously, a noexec /tmp would
cause an early mysterious UML crash.
UML really wants shared memory semantics form its physical memory map file,
and the place for that is /dev/shm. So move the default, and fix the error
messages to recognize that this value can be overridden.
I'm inconsistent enough about using dashes or underscores for punctuation
within filenames. However, sometimes I use both within the same name, and
I got sick of this. So, this fixes those cases.
This is the externfs/new hostfs/humfs patch. hostfs now seems to be stable.
The old hostfs will continue to exist until this one is as functional and
stable as it.
This fixes the interface of make_pipe, which doesn't need to initialize
filehandles. Instead, it is just a wrapper around pipe which just reclaims
descriptors if the initial call to pipe failes with -EMFILE.
Convert the boot-time host ptrace testing from clone to fork. They were
essentially doing fork anyway. This cleans up the code a bit, and makes
valgrind a bit happier about grinding it.
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
This patch is for testing only!
It's a quick and dirty patch to verify, that wrong fp-context
saving and restore really are the cause of errors in "memtest.c"
It fixes the problem by:
- making have_fpx_regs accessible for other modules instead of
declaring it "static" in arch/um/os-Linux/sys-i386/registers.c
- Adding 1 to HOST_FP_SIZE to have room for the "status" (and magic)
field. Now skas.fp + skas.xfp combined have the size of
struct _fpstatus
- in arch/um/sys-i386/signal.c adding some code, that handles
the _fpstatus in sigcontext differently, depending on have_fpx_regs.
For (have_fpx_regs == 0), the _fpstatus simply is copied to/from
user from/to skas.fp. When writing to user, the status field is
created also.
For (have_fpx_regs == 1), when writing to user, the full _fpstatus
is created in skas.fp and skas.xfp, from the data found in skas.xfp.
Then it is copied to user. When reading, the full _fpstatus is read
to skas.fp and skas.xfp. Then, skas.xfp is reconstructed from this
data.
X11 framebuffer driver from Gerd Knorr.
You have to enable CONFIG_FB (UML-specific options/Graphics support/
Support for frabe buffer devices), disable CONFIG_VGA_CONSOLE
(UML-specific options/Graphics support/Console display driver support/
VGA text console), and enable Framebuffer Console support (in the same
place), plus some fonts. You also seem to have to put 'x11=<width>x<height>
on the command line.
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
This patch adds vm_area structs for stub-code and stub-data.
So, stub-area is displayed in /proc/XXX/maps. Also, stub-pages
are accessible for debuggers via ptrace now.
Linux has a gate-vma concept, that unfortunately supports one
gate-vma only. Thus, there need to be done some changes in
mm/memory.c and fs/proc/task_mmu.c.
This patch avoids the mainline changes by using some dirty tricks.
So, the patch is for testing only, mainline should be changed
to support more than one gate-vma.
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
To support different subarches, UML must not use the same
address for jiffies and jiffies_64 in a hardcoded way.
I added JIFFIES_OFFSET to handle different arches. For
current arches, it is set to 0, for s390 it will be set to 4.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
s390 syscalls might be done by a "svc X" instruction (2 bytes
in size) or a "exec X,Y" instruction (4 bytes in size).
There is no way to read the size of the instruction via ptrace,
so UML/s390 can't do syscall-restarting by resetting instruction
pointer to the value before the syscall.
Also, in most cases syscall number is hardcoded in the "SVC X"
instruction, so there is no way to handle ERESTART_RESTARTBLOCK
correctly by *really* restarting the syscall.
s390 host has implemented TIF_RESTART_SVC-flag to handle the
latter case.
In UML we have to use TIF_RESTART_SVC for both cases.
This patch implements TIF_RESTART_SVC in UML.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
s390 syscalls might be done by a "svc X" instruction (2 bytes
in size) or a "exec X,Y" instruction (4 bytes in size).
There is no way to read the size of the instruction via ptrace,
so UML/s390 can't do syscall-restarting by resetting instruction
pointer to the value before the syscall.
Also, in most cases syscall number is hardcoded in the "SVC X"
instruction, so there is no way to handle ERESTART_RESTARTBLOCK
correctly by *really* restarting the syscall.
s390 host has implemented TIF_RESTART_SVC-flag to handle the
latter case.
In UML we have to use TIF_RESTART_SVC for both cases.
This patch implements TIF_RESTART_SVC in UML.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
s390 normally doesn't support a method, that allows us to force
the host to skip its syscall restart handling.
I implemented a new method in the host, which also is posted to
LKML to hopefully be inserted in s390 mainline.
To check availability of this change, I added a new check, which
is done in a slightly different way for the other arches, too.
Success in check_ptrace() and success in the new check are
absolutely necessary for UML to run in any mode.
So I changed the sequence of checks to:
1) check_ptrace() being called at startup very early
2) check_ptrace() calls the new check, too
3) can_do_skas() is called after check_ptrace()
check_ptrace() will never return, if it fails, but it now uses
printf() and exit() instead of panic().
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
s390 doesn't have PTRACE_GETREGS and friends, but has
PTRACE_[PEEK|POKE]USR_AREA to let user of ptrace() read or write
struct user as he wants.
So we need to support this operation conditionally.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
UML/s390 needs to reset FP control register in execve1, as
Linux s390 does.
I choose the way to use a macro, as this doesn't need any
changes in the other subarches.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
On UML/s390, signal-handlers are defined to have 4 parameters,
while the standard definition for signal-handlers only has one.
So we must not have prototypes of the handlers using one param
only in headers, that are included in the source, that defines
the handlers.
Just that conflict occurs with the incremental patches for
2.6.12-rc3. arch/um/os-Linux/signal.c defines boot_timer_handler
to have 4 parameters (ARCH_SIGHDLR_PARAM), while
arch/um/include/kern_util.h holds a prototype with one int as
param only.
With the latest patches, signal.c includes kern_util.h
indirectly via os.h: build fails.
So, I simply remove the prototype from kern_util.h and give each
modules calling it a separate prototype.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
This puts the int sig back in the signal handler declarations. Before,
with it in ARCH_SIGHDLR_PARAM, there was a use of sig, but no visible
declaration.
From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
In s390, fpregs are not reset in signal handlers.
Thus we may stop stub_segv_handler on s390 with a breakpoint instruction
instead of calling getpid, kill and sigreturn.
To make this run, we must not mask any signals in stub_segv_handler.
So I added conditional execution of set_handler in userspace_tramp
depending on ARCH_STUB_NO_SIGRETURN. If this macro isn't defined,
the code remains unchanged, else no signals for sa_mask are defined
and SA_NODEFER is added to flags.
Using the change, we also no longer need to care about correct stack
pointer for sigreturn, which would cause some nasty code on s390.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
SKAS0 and SKAS3 are tested a bit at least system boots and
shuts down correctly, network (tun/tap) works and we even could
start YaST on it.
Note:s
We use a host running SuSE SLES8 with a "private" kernel named
2.4.21-fsc.11, that contains special adaptions and drivers for
Fujitsu-Siemens mainframes. This means, our current SKAS3-patch
doesn't fit to vanilla kernels. We will create a reworked patch for
vanilla later (2.4 and 2.6).
Our 2.4 kernel also contains two fixes and one enhancement, that all
are essential to make UML run, even in SKAS0. The enhancement is to
support PT_TRACESYSGOOD, that generally is available in 2.6 kernels
but not in 2.4 for s390. So I would suggest to use 2.6 host for the
moment.
The fixes meanwhile are included into mainline. I don't know precisely
the first version containing them, in 2.6.13-rc5 they are present.
As those two patches are very small, they are inserted here as as
comment (AFAICS, its easy to do the changes by hand on older kernel
versions):
First patch to fix signal stack handling:
--- a/arch/s390/kernel/signal.c 2005-03-22 11:07:39.000000000 +0100
+++ b/arch/s390/kernel/signal.c 2005-03-22 11:08:44.000000000 +0100
@@ -285,7 +285,7 @@
/* This is the X/Open sanctioned signal stack switching. */
if (ka->sa.sa_flags & SA_ONSTACK) {
- if (! on_sig_stack(sp))
+ if (! sas_ss_flags(sp))
sp = current->sas_ss_sp + current->sas_ss_size;
}
Second patch to allow skipping of syscall restart:
--- a/arch/s390/kernel/ptrace.c 2005-05-07 07:20:31.000000000 +0200
+++ b/arch/s390/kernel/ptrace.c 2005-08-02 06:45:48.000000000 +0200
@@ -723,6 +761,13 @@
? 0x80 : 0));
/*
+ * If the debugger has set an invalid system call number,
+ * we prepare to skip the system call restart handling.
+ */
+ if (!entryexit && regs->gprs[2] >= NR_syscalls)
+ regs->trap = -1;
+
+ /*
* this isn't the same as continuing with a signal, but it will do
* for normal use. strace only continues with a signal if the
* stopping signal is not SIGTRAP. -brl
This patch changes the ubd I/O submission process to avoid some sleeping.
When the host returns -EAGAIN from io_submit, do_ubd_request returns to
its caller, saving the current state of the request submission in the
struct ubd. This state consists of the request structure and the range
of sg entries which have not yet been submitted. If the request queue is
drained, then this state is reset to indicate that, the next time it is
called, a new request needs to be pulled from the request queue.
When do_ubd_request returns because the host can handle no more requests,
it is necessary to rerun the queue after some completions have been handled.
This is done by adding the device to the restart list. ubd_intr walks
this list before returning, calling do_ubd_request for each device.
In addition, the queues and queue locks are now per-device, rather than
having a single queue and lock for all devices.
Note that kmalloc is still called, and can sleep. This is fixed in a
future patch.
This patch changes the ubd I/O submission process to avoid some sleeping.
When the host returns -EAGAIN from io_submit, do_ubd_request returns to
its caller, saving the current state of the request submission in the
struct ubd. This state consists of the request structure and the range
of sg entries which have not yet been submitted. If the request queue is
drained, then this state is reset to indicate that, the next time it is
called, a new request needs to be pulled from the request queue.
When do_ubd_request returns because the host can handle no more requests,
it is necessary to rerun the queue after some completions have been handled.
This is done by adding the device to the restart list. ubd_intr walks
this list before returning, calling do_ubd_request for each device.
In addition, the queues and queue locks are now per-device, rather than
having a single queue and lock for all devices.
Note that kmalloc is still called, and can sleep. This is fixed in a
future patch.
To ensure that I/O can always make progress, even when there is no
memory, we provide static buffers which are to be used when dynamic
ones can't be allocated. These buffers are protected by flags which
are set when they are currently in use. The use of these flags is
protected by the queue lock, which is held for the duration of the
do_ubd_request call.
There is an allocation failure emulation
mechanism here - setting fail_start and fail_end will cause
allocations in that range (fail_start <= allocations < fail_end) to
fail, invoking the emergency mechanism.
When this is happening, I/O requests proceed one at a time,
essentially synchronously, until allocations start succeeding again.
This currently doesn't handle the bitmap array, since that can be of
any length, so we can't have a static version of it at this point.
This patch completes the robustness and deadlock avoidance work by
handling the writing of the bitmap. The existing method of dealing
with low-memory situations by having an emergency structure for use
when memory can't be allocated won't work here because of the
variable size of the bitmap buffer, and the unknown (to me) limit of
a contiguous I/O request.
The allocation is avoided by writing directly from the bitmap rather
than allocating a buffer and copying the relevant chunk of the
bitmap into it.
This has a number of consequences. First, since the bitmap is
written directly from the device's bitmap, bits should not be set in
it until the I/O is just about to start. This is because reads
would see that and possibly race with the outgoing writes, returning
data from a section of the COW file which has never been written.
To prevent this, reads that overlap a pending bitmap write are
stalled until the write is finished. Modifying the bitmap bits as
late as possible shrinks the window in which this could happen.
Second, a section of bitmap that's being written out should not be
modified again until the write has finished. Otherwise, a bit might
be set and picked up by a pending I/O, resulting in it being on disk
too soon.
So, there are a couple new lists. Bitmap writes which have been
issued, but not finished are on the pending_bitmaps list. Any
subsequent bitmap writes which overlap a pending write have to
wait. These are put on the waiting_bitmaps list. Whenever a
pending bitmap write finishes, any overlapping waiting writes are
tried. They may continue waiting because they overlap an earlier
waiter, but at least one will proceed. Third, reads which overlap a
pending or waiting bitmap write will wait until those writes have
finished. This is done by do_io returning -EAGAIN, causing the
queue to wait until some requests have finished.
This patch completes the robustness and deadlock avoidance work by
handling the writing of the bitmap. The existing method of dealing
with low-memory situations by having an emergency structure for use
when memory can't be allocated won't work here because of the
variable size of the bitmap buffer, and the unknown (to me) limit of
a contiguous I/O request.
The allocation is avoided by writing directly from the bitmap rather
than allocating a buffer and copying the relevant chunk of the
bitmap into it.
This has a number of consequences. First, since the bitmap is
written directly from the device's bitmap, bits should not be set in
it until the I/O is just about to start. This is because reads
would see that and possibly race with the outgoing writes, returning
data from a section of the COW file which has never been written.
To prevent this, reads that overlap a pending bitmap write are
stalled until the write is finished. Modifying the bitmap bits as
late as possible shrinks the window in which this could happen.
Second, a section of bitmap that's being written out should not be
modified again until the write has finished. Otherwise, a bit might
be set and picked up by a pending I/O, resulting in it being on disk
too soon.
So, there are a couple new lists. Bitmap writes which have been
issued, but not finished are on the pending_bitmaps list. Any
subsequent bitmap writes which overlap a pending write have to
wait. These are put on the waiting_bitmaps list. Whenever a
pending bitmap write finishes, any overlapping waiting writes are
tried. They may continue waiting because they overlap an earlier
waiter, but at least one will proceed. Third, reads which overlap a
pending or waiting bitmap write will wait until those writes have
finished. This is done by do_io returning -EAGAIN, causing the
queue to wait until some requests have finished.
The pipe to the AIO thread needs to be non-blocking so that we know when
to return -EAGAIN and process some completions before trying some more
requests.
This patch eliminates the atomic count associated with a bitmap_io
struct. The original thinking was that there would be a number of
aio structures associated with the bitmap_io, since different chunks
of the sg element could go to different layers. The count was
needed to know when the full sg segment reached disk and it was safe
to write the bitmap.
However, the flaw in that thinking is that a bitmap_io struct is
only needed for writes, and writes always go to the COW layer.
Hence, there will only be one bitmap_io per aio, and the counting is
unnecessary.
This patch makes it possible to merge the aio and bitmap_io structs,
which would be a good cleanup.
I noticed that the common case in io_submit is an immediate context
switch to the AIO thread when it returns from io_getevents, followed
by a switch back. This patch changes that by having the AIO thread
wait on a pipe before calling io_getevents. When the kernel
finishes a batch of I/O, it writes the number of requests down the
pipe, and the AIO thread waits for that number, and goes back to
sleeping on the pipe.
This probably shouldn't reach mainline, as O_DIRECT I/O should have
the property of causing switching on every I/O request. Also, the
wakeup mechanism should be only used when the other side might be
sleeping.
I noticed that the common case in io_submit is an immediate context
switch to the AIO thread when it returns from io_getevents, followed
by a switch back. This patch changes that by having the AIO thread
wait on a pipe before calling io_getevents. When the kernel
finishes a batch of I/O, it writes the number of requests down the
pipe, and the AIO thread waits for that number, and goes back to
sleeping on the pipe.
This probably shouldn't reach mainline, as O_DIRECT I/O should have
the property of causing switching on every I/O request. Also, the
wakeup mechanism should be only used when the other side might be
sleeping.
This uses the C99 syntax to initialize an io_thread_req. Given how this
is compiled, it may not be a good idea, as it will consume more stack
than it should.
Clean up the jmpbuf code. Since softints, we no longer use sig_setjmp, so
the UML_SIGSETJMP wrapper now has a misleading name. Also, I forgot to
change the buffers from sigjmp_buf to jmp_buf.
Blairsorblade noticed some confusion between a system call's return
value and errno. This patch fixes a number of related bugs -
using errno instead of a return value
using a return value instead of errno
forgetting to negate a error return to get a positive error
code
Here's a list of interesting things that need doing, but which I'm not
going to have time for any time soon. These are all going to happen
after UML V1.0 hits the streets. If you're interested in working
on any of these and want pointers, help, etc, contact the
UML devel list.
Architecture Ports
UML needs to be ported to other Linux architectures
besides i386. This isn't very hard. The ppc port is nearly working
at this point, and that has provided a lot of information that other
ports will need. This is written up
here.
OS Ports
It also needs to be ported to other operating systems.
Other Unixes would probably be easiest. An OS needs some way of
intercepting Linux system calls in order for a port to be possible.
There are other needs, but they can probably be worked around if
they're not met. A Windows port would be most interesting. NT
apparently has the basic capabilities to run UML. Because UML
provides a secure virtual machine, running it on Windows would provide
a sandbox in which users could run untrusted code (like the email
virus du jour) safely. It would be very amusing to have Linux provide
a capability for Windows that Windows can't provide for itself.
It turns out that Michael Vines has written what amounts to the core
of a Windows port. It's a Windows app which intercepts Linux system
calls and interprets them itself. This is the critical part of UML.
If this works, everything else can probably be made to work. So, if
you're interested in working on this and want a head start, see
his
page for the details.
Native driver development
It is possible to allow direct access to hardware. This would allow native
drivers to run (and be debugged) in userspace. This requires that the host
provide access to io space (there is currently the ability to map io
memory into a process address space, which I think satisfies this),
and provide something to sit on the device irq and notify the
userspace driver of interrupts. I think this can be done with a stub
driver which probes the device at boot time, provides a file that the
userspace driver can open, and sends a SIGIO down the file descriptor
whenever the device interrupts. The thing I'm not sure about is
timing. If the driver needs to do stuff with interrupts disabled,
then that would make a userspace version hard, since processes can't
run without interrupts on.
As of 15 Apr 2001, it is possible to do USB
development under UML. Johan Verrept
posted
a patch which allows a USB
device driver in UML to control a physical USB device on the host.
hostfs extensions
hostfs is now used to access filesystems on the host, but it's
potentially much more general than that. It's separated cleanly into
kernel and userspace pieces, with the userspace code dealing with the
external data.
The simplest extension would be to move the user code to a remote
machine and put a simple RPC mechanism between the kernel and
userspace pieces. This would allow mounting of a remote directory
into a UML, similar to the userspace nfs daemon.
A more interesting possibility is to make the userspace host code deal
with non-filesystems that can be made to look like filesystems. There
are lots of databases which might be interesting to represent as
filesystems:
SQL databases - each row is a directory, with each column a file
containing the entry for that row and column
File databases like /etc/passwd and /etc/group - each entry is a
directory with columns as files like the SQL scheme above
Network databases like NIS and BIND
Other possibilities include
networks - with each machine getting a directory containing
information about that box
people and users
And there are probably lots of other possibilities. Anything that can
at all be reasonably made to look like a file or filesystem could
be mounted inside UML.
To make this work, all that's needed is a rewrite of
arch/um/fs/hostfs/hostfs_user.c. It implements the operations on the
underlying objects, which currently are host files. If you can
implement those same operations for some other kind of object, then
that object, or a set of them, can be mounted inside a UML and
operated on as files.
Storing the same data in multiple places and mounting them jointly as
a single hosfs filesystem offers other possibilities as well:
Dumping a directory into a filesystem into a database and having
hostfs provide access to both would allow normal file access to the
filesystem plus queries to the database through a mechanism such as an
ioctl or a virtual filesystem on the side. So, you'd get both normal
access to the filesystem plus the ability to do database queries on it.
Mounting many almost-identical directories from multiple machine on a
single hostfs mount point would allow software to be installed to all
the machines simulataneously with a single install command inside the
virtual machine.
SMP support
Enabling SMP in UML would allow it to emulate a multiprocessor box.
Getting this working should be fairly straightforward. I did have it
working in early 2000, but turned it off to work on the UP side of
things.
Here's what needs to be done:
Uncomment the SMP config in arch/um/config.in, run your favorite
config, and turn it on
Get UML to compile - there's been some bitrot since this last worked,
so the some of the kernel headers don't agree any more with what's in
UML, and I've put in some #error directives where I've noticed there
needs to be some thought about SMP.
Audit the UML arch code, looking for global data that needs locking.
There isn't very much of it. I did the audit in a couple of days, and
didn't miss very much.
Debug it. Bang on it until you stop hitting races.
Send in the patch.
UML Clustering
DSM stands for Distributed Shared Memory, where the nodes of a cluster
share a portion of their memory through some kind of special
interconnect.
This can be done with UML because UML's physical memory is really
virtual memory on the host, so it can be mapped and unmapped. So, the
idea is to spread a single UML instance over multiple hosts by running
a virtual processor on each host and partitioning UML physical memory
between them. A page that's present on one node will be unmapped from
the others. If one of the other nodes accesses it, it will fault, and
a new low-level fault handler will figure out what node currently has
it, and request that it be sent over. The other node will unmap it,
and copy it over to the requesting node, which will map it in to the
appropriate location and continue running.
This will get a UML-based Single-System Image (SSI) cluster up and
running. It will be very inefficient because there are data
structures that the kernel references very frequently. The pages that
contain this data will be constantly being copied from node to node
and the nodes will be spending much of their time waiting for that to
happen.
There are two avenues through which this can be fixed. One follows
from the observation that this cluster is an extreme form of NUMA,
with no global memory and very expensive access to other nodes' local
memory. So, the ongoing NUMA work going into Linux will help this.
Plus, UML will effectively make NUMA hardware available to everyone
who runs Linux, which will hopefully pull more people into the effort
and speed it up.
The other is to start replacing the shared memory communication with
an RPC-based communication mechanism. While the NUMA work will reduce
the amount of inter-node chatter, an RPC mechanism will make the
remaining communication much more efficient. The ultimate outcome of
this effort will be a standard SSI cluster implementation which can
run both with physical nodes or virtual ones.
UML as a normal userspace library
UML normally runs as a standalone executable, but there's no reason
bthat it can't be packaged slightly differently, as a library which
other applications could link against. Those apps would gain access
to all the functionality in the kernel, including
memory allocation - these are specialized to be fast and scalable.
In addition the virtual memory allocator is able to defragment memory
efficiently with a buddy system algorithm.
memory management - built-in swapping and cache size control. With
some work, the userspace swapping code could be used to tell the host
kernel what memory the application doesn't really need, allowing
userspace and kernelspace to cooperate in getting the best performance
from the system.
virtual memory and multiple address spaces - these are things which
libc doesn't provide, except by calling into the kernel to get them.
They can be used to jail some of the application's threads, if it is
running something it doesn't trust. With the addition of highmem
support to UML, this could be used to transparently provide access to
more than 4G of address space.
threads - these are specialized to be low-overhead and fast, as is the
scheduler.
filesystems - the filesystems can be thought of hierarchical storage
systems for the application's internal data. They can be mounted on
an internal ramdisk, making them private and non-persistent, or an a
host file, making them persistent and possibly shared.
the full network stack and network devices - this is interesting when
combined with the presence of a filesystem inside the application. It
could export the filesystem via NFS or another remote filesystem
protocol to the outside world.
Here are some examples of how these capabilities can be used:
Apache could store its configuration in an internal filesystem and
export it to the outside world over the network. All of the Apaches
in a server farm could store their configurations in an internal
filesystem and export them to the outside world over their networks.
A central server mounts them all and uses information about the load
on each machine to tweak the settings of its Apache so that the server
does as much work as possible without causing it to be overloaded.
Alternatively, the Apaches could all import their configurations from
a central location so that when a change is made on the central
server, all of the servers immediately see the change. This would be
useful for things like virtual server configuration. Create the new
virtual server configuration by creating the appropriate directories
and files in the configuration filesystem and all of the web servers
importing that configuration would immediately start serving the new
virtual domain.
Combine the two previous examples, with each server exporting
performance-related configuration items so they can be individually
tweaked on the fly, but importing the global configuration items that
need to be shared across the entire server farm.
These servers could use the virtual memory and jailing abilities of
UML to accept code from web clients and run it. These pieces of code
would be run as UML processes, jailing them inside restricted
environments with their own private address spaces, allowing them to
change their own data, but providing no access to any other data. To
prevent these threads from consuming too much CPU and preventing the
servers from doing their normal work, they could be given a lower
priority than the normal Apache threads, so that the UML scheduler
will only run a client's thread when the server would otherwise be idle.
An interactive application could store its UI in an internal
filesystem and export it to the host. External scripts could then
navigate around that filesystem, activating parts of the interface,
which would effectively provide scripting to applications which don't
have it built-in.
A script could watch the exported filesystem for events such as a dialog
box being opened and fill in the fields with defaults from information
sources that the application knows nothing about. The script could
simply remember the last set of values for that dialog box and fill
them in the next time. Or it could be watching a number of
applications, copying information from one to the other. For example,
if the user has just looked up a particular person's calendar and is
now starting to compose an email message, the script could look up
that person's email address in its own database, and put it in the
'To' field of the new message composition window.
Clusterized applications
Combine the ideas of using UML as a cluster and using it as an
application library and you get an application which can spread itself
over multiple machines and effectively remain a single process. Since
the kernel's data structures and algorithms will preseumably be suited
for clustering already, the more extensively the application uses them
for its own purposes, the more prepared it will be to spread itself
transparently over a virtual cluster.
Servers would be able to automatically add virtual nodes on new
physical hosts to themselves as the load increased and shut them down
as their load goes back down. And as they do that, some other
clusterized server may be starting to run on those hosts as its load
goes up. This will makes it possible to run one server for each
service on a server farm, rather than one server per machine.
You may want to resize a UML filesystems because it isn't large
enough. This is a common complaint with the filesystems available for
download here since they tend to be just big enough to hold what's
pre-installed in them, making it tough to install more stuff.
Shut down the uml guest
UML# halt
Make a backup of the file you want to resize.
If you have any COW files which have the file you want to resize as a
backing file then make a backup of the cow files, too.
If you want to merge the cow file with uml_moo, do it now. Otherwise
remove the cow file to avoid confusion.
Make sure the filesystem is in a clean state.
host% e2fsck -f filename
Resize the file.
host%
dd if=/dev/zero of=filename bs=1 count=1 seek=newsize conv=notrunc
0+0 Records in
0+0 Records out
newsize can be any size recognized by "dd", such as "256M",
"4G", or "2T". The length of the file should have changed, but not
the actual filesize as with "ls -ls".
Resize the filesystem.
host% resize2fs -p filename
If you don't trust your luck and/or your computer run:
Here, I describe my view of the future of UML. It is almost always
seen as a standard virtual machine technology, since that is the only
way it can be currently used, but, because of its design as a virtual
OS rather than a virtual machine, it is a general-purpose
virtualization technology which can be put to a much wider array of
uses.
I'm dividing the descriptions below roughly by their timescale,
starting with ongoing stuff which will continue for the forseeable
future, and ending with things which are so far into the future that
they depend on enabling technologies which don't yet exist for Linux.
Ongoing work
Work that has been happening, and will continue to happen for the
foreseeable future include:
bug fixes
functional enhancements - Small features will go in from ocassionally
as time permits to implement them, or as they are implemented by other
people who send me patches. Examples of this include:
Finishing the hotplug support, including adding CPU and memory hotplug
Integrating
sysemu
support, and pushing the host patch to Linus
Adding support for valgrind, including describing the memory
allocators so it can track kernel memory usage
keeping up with 2.4, 2.6, and -mm - UML will be updated to the latest
2.4 and 2.6 trees, with 2.4 maybe being dropped when it's clear that
the world has moved on to 2.6. I will pick up 2.7/2.8 when they
start. I'm also tracking the -mm kernels as a staging area for stuff
that destined for Linus.
Performance improvements - Profiling both the host and UML, looking
for ways to make UML more efficient as a guest and to make the host a
better host.
Externfs
externfs is the result of restructuring hostfs to better support
mounting arbitrary outside resources as UML filesystems. Currently,
there are two users of it, hostfs and humfs, both of which import host
directory hierarchies as UML filesystems. However, externfs allows
anything which can implement a standard set of file operations to be
mounted as a UML filesystem.
For example, a SQL database could be imported as a filesystem into a
UML. The sqlfs plug-in to externfs would define the mapping from
tables, rows, and columns to UML directories and files. Reads from
these files would trigger SQL lookups, and writes to them would
trigger updates. Since the data is stored in the database, the
database and its indexes will always be up to date.
Also, A special file (or directory) could be made
available which would allow SQL access to the underlying database.
So, if /mnt is a sqlfs mount,
ls /mnt/sql/"select * from table where foo=3"
could list the set of rows matching the select criteria, and changing
to that directory and looking at the files in it could show the data
in those rows.
I see this particular case as being more of a cute trick than
something that will be generally useful. However, a special case
could be very useful. If a Linux filesystem were stored in the
database, UML could boot from the sqlfs filesystem, and then the raw
SQL access would start being useful. Assuming that file metadata was
stored reasonably in the database, you could do queries such as
ls /sql/"select name from files where setuid=1"
ls /sql/"select name from files where modified=today"
ls /sql/"select name from files where created=today
and owner="root"
These have obvious security uses, and can be done casually and
quickly, since the file metadata is indexed in the underlying
database, rather than having to run a find over the entire
filesystem.
This idea could be extended to mounting specialized databases
containing certain types of data. For example, a UML user could store
email in a filesystem which is backed by a glimpse database on the
host. This would allow rapid searching of email using whatever
searching capabilities the filesystem makes available inside UML.
Medium-term projects
For some time, I have been planning to do away with the double-caching
that UML causes on the host. When UML reads data, it does so in a way
that the data is stored in the host's page cache. That data also ends
up separately in the UML's page cache, so that there are two copies of
the data in the host's memory, which is an obvious waste. My solution
to this is two-fold:
use O_DIRECT IO on private data
mmap shared data into the UML page cache
Using O_DIRECT IO bypasses the host's page cache and causes the data
to be read (or written) directly into (or from) the UML's page cache.
For data which is private to the UML, this is perfect. However, if
there's shared data, such as a backing file with multiple COW files,
this will create one copy of the data in each UML which uses it. The
host copy is eliminated, which is an improvement, but it's possible to
do better in this case.
For shared data, the solution is to let the data go through the host's
page cache, and have the UMLs mmap it from there. This eliminates the
copying since all the UMLs have the same page of memory mapped into
their page caches.
This could obviously be done in the case of private data as well, but
mmap is expensive - mapping a page seems to be somewhat slower than
copying it. So, using mmap in the case of private data would just
slow it down. Of course, this is true of shared data as well, but
there are offsetting gains in the form of memory reductions, and
speedups for the second and later UMLs which access the data.
A related but separate project is the use of the AIO support in 2.6
hosts to improve UML's IO performance. humfs has AIO support, and
will use it if it's present on the host. This needs to be extended to
include the ubd driver and hostfs.
ports
swsusp
UML in the kernel
It is possible to configure UML to run inside the host kernel rather
than in userspace. This involves making it use internal kernel calls
rather than the libc system call wrappers. This would turn UML into a
kernel module.
The most obvious advantage to doing this is speed. It would eliminate
a lot of the context switching that UML currently has to do. In
particular, it could greatly reduce the UML system call overhead.
Personally, I don't favor moving UML into the kernel just for
performance - there are much better reasons for doing it. My goal is
to get UML's userspace performance close enough to the hardware that
it would require measurement to tell the difference.
That being said, comparing the performance of a kernel UML instance
and a userspace instance would indicate where the overhead is, and
possibly point at new interfaces which would allow a userspace
instance to achieve performance closer to a kernel instance.
There are better reasons for loading UML into the kernel:
Direct hardware access
Resource control
Giving UML the ability to directly control hardware would allow
partitioning the server between UML instances. In the most extreme
case, the host kernel would be not much more than a hypervisor which
just starts the UML instances, gives them hardware, and gets out of
the way. This would be a sort of soft partitioning, which wouldn't
require hardware partitioning support, but would also not provide the
protection offered by hardware support.
So, there's the full range of configurations, from fully virtualized
hardware, as with a default UML instance in the kernel, to fully
physical hardware, as just described. Also, any combination of
physical and virtual hardware. So, two instances could each be given
half of the physical memory (minus whatever the host needs), and half
the disks, but if there's only one physical NIC, then the host would
retain control of that, and provide each instance a virtual NIC in the
form of a TUN/TAP device.
Once UML can be loaded into the kernel, the next step will be to break
it into pieces, so that a subsystem can be loaded by itself, without
the rest of UML. These guest subsystems can be used as compartments
in which processes can be restricted to whatever resources were
provided to the compartment.
For example, loading a guest VM system into the kernel and giving it
512M of memory to manage, and then sticking some processes in it will
restrict those processes to that 512M. When they start using up that
memory, the guest VM system will start swapping or dropping clean
pages in order to free memory within the compartment. It will do even
if there is plenty of free memory on the system outside the
compartment.
Similarly, a guest scheduler can be used to restrict the CPU usage of
a set of processes, and a guest network stack can be used to restrict
their network bandwidth.
Embedded UML
Going back to userspace, the next step is to make UML configurable as
a captive virtual machine. By this, I mean UML would be linked into
an application, which would gain an internal virtual machine for its
own use.
There are a number of possible uses for a captive UML:
a general-purpose internal programming environment
a platform for managing a service from the "inside"
an internal authentication and authorization service
I'll use Apache to illustrate these, but they also apply to other
services and applications. Apache already has internal scripting
environments in the form of mod_perl and mod_python, which embed Perl
and Python interpreters inside Apache, and probably others. An
embedded UML could be used for the same purpose by installing the
normal Perl or Python environments inside it, and running the scripts
there, rather than inside Apache itself.
This handles the actual execution of the scripts, but we need a way of
communicating between Apache and the scripts inside the UML. Apache
has to pass HTTP requests in, and receive HTTP responses back. One
way of doing this is with an Apache-specific (call it apachefs)
filesystem which is mounted inside the UML. Reads, writes, and other
file operations within this filesystem would be implemented by Apache.
So, one of the files would be read by the script inside the UML, and
whenever an HTTP request came in, it would be parsed, and sent to the
script via this special file. The script would block reading this
file until a request came in for it to handle. When it had the
results, it would write them back to Apache through a similar (and
possibly the same) file.
This has a bunch of advantages over the current mod_perl/mod_python
arrangement:
The scripts are jailed, so they can't access the host, and their
resource consumption is restricted. This is a major advantage, since
anyone who wants to have a dynamically generated web site needs a
dedicated server for it. The less expensive shared server
arrangements offer, at best, cgi. This would allow dynamic HTML from
within a shared server. There would be a UML instance for each such
web site, but they could be a fairly small ones, so they wouldn't
necessarily increase host resource consumption drastically.
The scripts can be written in any language and run in any environment
supported by Linux. There doesn't need to be an appropriate mod_foo
for Apache, just the ability for Linux to execute the script.
The scripts can be monitored, debugged, and modified from inside the
live server by anyone who can ssh in to the UML. Of course, doing
this on a production server is risky, but the capability would be
there when it's really needed.
Once there's a way of communicating HTTP requests and responses back
and forth between Apache and something running inside its captive UML
via apachefs, it's relatively easy to communicate other information
back and forth via other files within that filesystem. In particular,
it could be used as something analogous to procfs for the kernel.
procfs is a window on the kernel's internal state, and procfs files
can be used to read and modify that state. Similarly, apachefs could
be used as a window on Apache's internal state. Statistics would be
available from read-only files, and configuration variables would be
available from read-write files.
This would allow the Apache user to configure that web site from
within Apache. This would obviously need to be controlled so that
only variables relevant to a single web site could be configured from
within that site's captive UML. Doing this would relieve the host's
system manager from handling configuration changes.
If this was done on a server running multiple services, and all the
services had this "management from the inside" arrangement with a
captive UML, then the server's admin wouldn't normally need to deal
with the services. The managers of the services wouldn't even need
accounts on the server - all they need is an account on their service's
captive UML. So, this would be a nice security arrangement for the
server.
Finally, just as we can load UML subsystems into the kernel with
useful results, we can link UML subsystems into applications and get
something that's similarly useful. Consider linking a captive VM
system into an application, which then uses it for its memory
allocation. The biggest difference between this and malloc is that
the Linux VM system is restricted to using a certain amount of memory,
and won't go beyond that. In contrast, malloc will allocate as much
memory as the system will allow. When the Linux VM system starts
running out of free memory, it will start releasing memory that's not
needed any more. In the context of an application, that means that
its memory consumption is limited, and when it starts approaching that
limit, it will start swapping.
This can make the application much better behaved with respect to the
system. If it starts getting overloaded, it will slow down, but it
won't affect the rest of the system and whatever else is running on
it. If everything else on the system is similarly configured, then
the memory usage will be very predictable. There won't be any
possibility of one service going crazy and grabbing all the memory on
the system.
As another example, consider linking a filesystem into an application
which uses it to store its internal state. The filesystem would flush
this data out to storage on the host, and the consistency semantics
provided by the popular filesystems would guarantee a level of
consistency for this data in the event of an application crash. So,
it could resume after the crash more or less where it was when it
crashed. This would provide automatic checkpointing, so that it could
be shut down in a controlled way, and resumed later from where it left
off.
Application clustering and SMP with UML
As Linux is made to run well on increasingly large systems, its SMP
scalability is improving. Large multi-threaded applications have many
of the same problems as an OS kernel running on a large SMP system.
So, the efforts made to improve the SMP scalability of the Linux
kernel can benefit a multi-threaded application which links against
UML and uses whatever kernel facilities it can.
Similarly, some applications are going to need built-in clustering.
An example of this is Oracle clustering with its ocfs filesystem. As
the Linux kernel gains clustering abilities, they will automatically
become available to applications which wish to use them via UML. If
an application needs to distribute its data across many instances of
itself, it can link against whatever UML subsystems it needs -
minimally, a cluster filesystem such as gfs or ocfs and the network
stack. The instances will then store their data in the filesystem,
which will take care of the clustering.
Hosted at
user-mode-linux-doc-20060501/run.html 0000644 0000000 0000000 00000033645 12742461304 014147 0 ustar
Running User Mode Linux
It runs on 2.2.15 or later, and all 2.4 and 2.6 kernels.
Booting UML is straightforward. Simply run 'linux': it will try to
mount the file `root_fs' in the current directory. You do not need to
run it as root. If your root filesystem is not named `root_fs', then
you need to put a `ubd0=root_fs_whatever' switch on the linux command
line.
You will need a filesystem to boot UML from. There are a number
available for download from here
. There are also several
tools which can be used to generate UML-compatible filesystem
images from media.
The kernel will boot up and present you with a login prompt.
Note: If the host is configured with a 2G/2G address space split
rather than the usual 3G/1G split, then the packaged UML binaries will
not run. They will immediately segfault. See
this page for the scoop on
running UML on your system.
Logging in
The prepackaged filesystems have a root account with
password 'root' and a user account with password 'user'. The
login banner will generally tell you how to log in. So, you log in
and you will find yourself inside a little virtual machine. Our
filesystems have a variety of commands and utilities installed (and it
is fairly easy to add more), so you will have a lot of tools with
which to poke around the system.
There are a couple of other ways to log in:
On a virtual console
Each virtual console that is configured (i.e. the
device exists in /dev and /etc/inittab runs a getty on it) will come up
in its own xterm. If you get tired of the xterms, read
this to see how to
attach the consoles to something else, like host ptys.
Over the serial line
In the boot output, find a line that looks like:
serial line 0 assigned pty /dev/ptyp1
Attach your favorite terminal program to the corresponding
tty. I.e. for minicom, the command would be
host% minicom -o -p /dev/ttyp1
Over the net
If the network is running, then you can telnet to the virtual machine
and log in to it. See the
networking page to learn about setting up a virtual network.
When you're done using it, run halt, and the kernel
will bring itself down and the process will exit.
Each month, we will introduce a disaster scenario and accept
submissions of recovery procedures from them. For examples of good
disasters, see my O'Reilly article
here.
If you're new to UML, you'll probably want to read the following pages
before starting to solve this month's problem:
This month's disaster involves a
filesystem that
mysteriously won't boot. Your job is to figure out what's wrong and
fix it. This one is fairly easy, so if you find it to be trivial,
don't complain.
Anyway, download the filesystem, uncompress it, make it boot, and tell
us below how you did it.
Submit your solution
If you have a solution to this month's problem and you want it to be
immortalized on this very site, submit it here.
I will pick one or more winning solutions based on criteria such as
originality - all else being equal, I like non-obvious solutions
subtlety - if applicable, small fixes are better than big ones
brevity - short and sweet is better than long and involved
parsimony - the fewer external resources you need, the better
Propose a disaster
If you have a scenario which you think would make a good Disaster of
the Month, please submit it here. If you have a good solution,
include it as well. Disasters which have actually happened in real
life are especially good, but anything which can happen on a physical
box is fine.
Each month, I will look over the submissions and choose an interesting
one to feature as that month's disaster of the month.
Last Month's Disaster
The December, 2001
involved zeroing the root filesystem superblock and attempting to
recover it. (Note : I am lame, but not so lame that I don't realize
that December, 2001 is not the month preceding May, 2002. However, I
am lame enough to have run out of disaster ideas after writing the
O'Reilly article and starting SDOTM. So, this page just sat
here sadly until Roger Binns had pity on it and sent me some more
disaster ideas, one of which is the May contest.)
Last Month's Solutions
A number of proposed solutions involved reinstalling or restoring from
backups. These were rejected on the basis of being overly
heavy-handed. Most of the rest of the valid answers involved
UML# e2fsck -n alternate superblock number
I
tossed out the ones that suggested using 8193 as the alternate
superblock. That didn't work for me, and it turns out to be dependent
on the filesystem block size.
So, I kept the answers that provided a means of determining the
superblock location either from the filesystem itself or from the
block size. The earliest such answer came from nicholasperez (at) (a
VERY impolite domain name):
Similar answers came from mgalgoci (at) parcelfarce (dot) linux (dot)
theplanet (dot) co (dot) uk, MonMotha, Phil, skepticman, bluebird (at)
dartmouth (dot) edu, and tjw.
In addition, I decided to name some honorable mentions:
Solutions which involved fixing the filesystem on the host lost points
because you can't do that with a lost superblock on a physical system,
but dan_a (at) gmx (dot) net submitted a solution which pulled the
filesystem apart and reassembled it with the fixed superblock.
Petru Paler broke the rules slightly and submitted an InstaFix (tm)
which assumes that you realize immediately what you did, and that dd
is present in the page cache:
This lost points for hardcoding 8193, but gained some back for
possibly fixing the problem immediately.
willmore suggested
UML# mke2fs -S /dev/ubd/0
followed by
an fsck. This will cause mke2fs to attempt a filesystem rebuild. The
e2fsck will leave a bunch of files in lost+found, which you will have
to poke through to identify. Any directories shouldn't be hard, since
you can identify them through their contents, but the normal files
could be a pain.
Don't attempt to share filesystems simply by booting two UMLs from the
same file. That's the same thing as booting two physical machines
from a shared disk. It will result in filesystem corruption.
Using layered block devices
The way to share a filesystem between two virtual machines is to use
the copy-on-write (COW) layering capability of the ubd block driver. As of
2.4.6-2um, the driver supports layering a read-write private device
over a read-only shared device. A machine's writes are stored in the
private device, while reads come from either device - the private one
if the requested block is valid in it, the shared one if not. Using
this scheme, the majority of data which is unchanged is shared between
an arbitrary number of virtual machines, each of which has a much
smaller file containing the changes that it has made. With a large
number of UMLs booting from a large root filesystem, this leads to a
huge disk space saving. It will also help performance, since the host
will be able to cache the shared data using a much smaller amount of
memory, so UML disk requests will be served from the host's memory
rather than its disks.
To add a copy-on-write layer to an existing block device file, simply
add the name of the COW file to the appropriate ubd switch:
ubd0=root_fs_cow,root_fs_debian_22
where 'root_fs_cow' is the private COW file and 'root_fs_debian_22' is
the existing shared filesystem. The COW file need not exist. If it
doesn't, the driver will create and initialize it. Once the COW file
has been initialized, it can be used on its own on the command line:
ubd0=root_fs_cow
The name of the backing file is stored in the COW file header, so it
would be redundant to continue specifying it on the command line.
Note!
When checking the size of the COW file in order to see the gobs of
space that you're saving, make sure you use 'ls -ls' to see the actual
disk consumption rather than the length of the file. The COW file is
sparse, so the length will be very different from the disk usage.
Here is a 'ls -l' of a COW file and backing file from one boot and
shutdown:
host% ls -l cow.debian debian2.2
-rw-r--r-- 1 jdike jdike 492504064 Aug 6 21:16 cow.debian
-rwxrw-rw- 1 jdike jdike 537919488 Aug 6 20:42 debian2.2
Doesn't look like much saved space, does it? Well, here's 'ls -ls':
host% ls -ls cow.debian debian2.2
880 -rw-r--r-- 1 jdike jdike 492504064 Aug 6 21:16 cow.debian
525832 -rwxrw-rw- 1 jdike jdike 537919488 Aug 6 20:42 debian2.2
Now, you can see that the COW file has less than a meg of disk, rather
than 492 meg.
Another warning
Once a filesystem is being used as a readonly backing file for a COW
file, do not boot directly from it or modify it in any way. Doing so
will invalidate any COW files that are using it. The mtime and size
of the backing file are stored in the COW file header at its creation,
and they must continue to match. If they don't, the driver will
refuse to use the COW file.
If you attempt to evade this restriction by changing either the
backing file or the COW header by hand, you will get a corrupted
filesystem.
Among other things, this means that upgrading the distribution in a
backing file and expecting that all of the COW files using it will see
the upgrade will not work.
Moving a backing file
Because UML stores the backing file name and its mtime in the COW
header, if you move the backing file, that information becomes
invalid. So, the procedure for moving a backing file is
Move it in a way that preserves timestamps. Usually, this is a "-p"
switch. "cp -a" works because "-a" implies "-p".
Update the COW header by booting UML on it, specifying both the COW
file and the new location of the backing file
host% ubd0=COW file,new backing file
location
UML will notice the mismatch between the command line and COW header,
check the size and mtime of the new backing file path, and update the
COW header to reflect it if it checks out.
If you forget to preserve the timestamps when you move the backing
file, you can fix the mtime by hand as follows
host%
mtime=whatever UML says mtime should be ; \
touch --date="`date -d 1970-01-01\ UTC\ $mtime\ seconds`" backing file
Note that if you do this on a backing file that has truly been
changed, and not just moved, then you will get file corruption and you
will lose the filesystem.
uml_moo : Merging a COW file with its backing file
Depending on how you use UML and COW devices, it may be advisable to
merge the changes in the COW file into the backing file every once in
a while.
The utility that does this is uml_moo. Its usage is
host% uml_moo COW filenew backing file
There's no need to specify the backing file since that information is
already in the COW file header. If you're paranoid, boot the new
merged file, and if you're happy with it, move it over the old backing
file.
uml_moo creates a new backing file by default as a safety measure. It
also has a destructive merge option which will merge the COW file
directly into its current backing file. This is really only usable
when the backing file only has one COW file associated with it. If
there are multiple COWs associated with a backing file, a -d merge of
one of them will invalidate all of the others. However, it is
convenient if you're short of disk space, and it should also be
noticably faster than a non-destructive merge. This usage is
host% uml_moo -d COW file
uml_moo is installed with the UML deb and RPM. If you didn't install
UML from one of those packages, you can also get it from the
UML
utilities tar file in tools/moo.
uml_mkcow : Create a new COW file
The normal way to create a COW file is to specify a non-existant COW
file on the UML command line, and let UML create it for you. However,
sometimes you want a new COW file, and you don't want to boot UML in
order to get it. This can be done with uml_mkcow, which is a little
standalone utility by Steve Schnepp.
The standard usage is
host% uml_mkcow new COW fileexisting
backing file
If you want to destroy an existing COW file, then there is a -f switch
to force the overwriting of the old COW file
there is a special thread, the tracing thread, which does system call
tracing on the UML processes
the tracing thread nullified system calls, and caused the process to
enter the UML kernel, which is mapped into the upper part of its
address space
This was forced by the limited support for UML in Linux and has a
number of problems
The UML kernel is present in the address space of each of its
processes, and, by default, is writeable. This is obviously a
security problem, since, with write access to kernel data, a process
can break out to the host. UML's 'jail' mode fixes this problem by
making UML data readonly while a process is running, but this imposes
a huge performance penalty. Also, the kernel is still there, and can
be read, so this isn't acceptable for honeypots, since a bad guy can
easily tell that the system is a UML.
UML uses signals to force control to the UML kernel during a system
call or interrupt. Signal delivery and return are slow, and impose a
noticable performance hit.
During the fall of 2002, I started fixing these problems by adding
support in the host Linux for features that would solve these
problems. The result is now stable, and is being used by a number of
people with good results.
In short, the changes cause the UML kernel to run in an entirely
different host address space from its processes. This solves the
security and honeypot fingerprinting problems by making the UML kernel
totally inaccessible to UML processes. Their address spaces are
identical to what they would be on the host. This also provides a
noticable speedup by eliminating the signal delivery that used to
happen for every UML system call.
The old mode is now called 'tt mode', for Tracing Thread, and the new
mode is called 'skas mode', for Separate Kernel Address Space.
Using skas mode
Skas mode requires that a patch be applied to the host kernel. This
patch implements the address space support needed by skas mode, plus
some additions to ptrace which are necessary. The patch is available
from the UML
download page.
These patches are against 2.4.19, but are fairly non-intrusive, and
should apply to any nearby kernel. You should get the latest host skas
patch and run the latest UML on it. You should only use an older
host skas patch if you need to run an older UML for some reason.
With the patch applied to the host, all you need to do is make sure
that CONFIG_MODE_SKAS is enabled in UML, and run it. It will detect
the host support and use it. If the host support isn't there, it will
fall back to tt mode. It will also fall back to tt mode if it doesn't
detect a specific version of the host skas support. It will tell you
during the early boot which version it's looking for:
Checking for the skas3 patch in the host...found
Checking for /proc/mm...found
Once you see this, you're all set. UML is happily running in skas
mode.
Skas effects on UML
This is simple - you shouldn't really notice any difference, except
for performance. Inside UML, skas mode causes no changes in
functionality. For most uses, you should see a noticable improvement
in performance. My favorite benchmark, a kernel build, is almost
twice as fast with skas mode as with tt mode, and is within 30% of the
host's time. Bill Stearns saw a script's running time drop from ~50
seconds to ~14 seconds, almost a quadrupling in speed.
Skas effects on the host
You will notice some differences on the host. If you run ps, you will
notice only four processes per UML rather than the dozens that you see
in tt mode. They are
The UML kernel thread, which runs in the separate kernel address
space, executes kernel code, and does system call interception on UML
processes
The UML userspace thread, which runs all UML process code and switches
between host address spaces on each UML context switch
The ubd driver asynchronous IO thread
The write SIGIO emulation thread
If you are running many UMLs on the host, and you convert them over to
skas mode, you will also notice a large reduction in the host load.
The UML server that I am most familiar with, which runs 25-30 UMLs at
any given time, reduced its typical load average from 2 - 5 with tt
mode to .1 - .5 with skas mode.
Skas present
I know of only one bug in the current skas patch, which only affects
running nested UMLs as far as I know. I have been running the skas3
patch continuously since the fall with no problems. Other people
have, as well, and I've heard of no problems from them. It's
SMP-safe, and it's running stably on SMP hosts.
Skas future
There is going to be a skas4 patch at some point. There will be no
real functional changes. The bug mentioned above will be fixed, and a
performance tweak or two will be added.
The major change will be the interface to the new stuff. Currently,
new address spaces are created by opening /proc/mm, and they are
modified by writing to the returned file descriptor. Linus hates this
interface (and I wasn't very fond of it, either). After some
discussion, he proposed adding two new system calls, one which creates
new address spaces, and another which executes an arbitrary system
call in the context of one of these address spaces.
The skas4 patch will implement these new system calls, and UML will
use them at that point. After that, when the patch has had some
testing, I will think about how and when to get it into the main
kernel pools.
This is the README from the uml Slackware installation kit. The kit
contains a root filesystem which will run the installation and this
README. You can see a transcript of an installation
here.
inst_fs_slackware_7.0 contains a filesystem that the user-mode kernel
can use to generate a Slackware root filesystem, given access to a
Slackware distribution. This has been tested for Slackware 7.0. If
the Slackware install procedure is reasonably stable, it might work
for other releases, as well. If you try it on a different release,
let me
know whether it worked or not, and what, if anything, needed to be
done different from the instructions below.
To run the installation procedure, do the following:
Get a user-mode kernel. Any kernel from the user-mode linux download
page will do.
Make a file to contain the Slackware installation:
The "100" is the size in megabytes of the filesystem - Choose that
number according to what you want to install. These are the
approximate installation sizes of the various series:
See README7.TXT at the top level of the distribution for more details.
Make a filesystem in that file:
host% mke2fs root_fs_slackware
Run the kernel as follows:
host% linux ubd0=inst_fs_slackware_7_0 ubd1=root_fs_slackware
ubd2r=/dev/cdrom devfs=nomount rw
Replace "/dev/cdrom" with the path to the Slackware distribution CD on
your system. It can be the CD-ROM device file or a file that contains
the ISO image of the CD. It cannot be a directory containing the
CD-ROM directory tree. I.e. if your CD-ROM device is /dev/cdrom and
it's mounted on /mnt/cdrom, pass /dev/cdrom on the linux command line,
not /mnt/cdrom.
You will get four xterms popping up, one of which contains the
system log, and the other three of which contain login prompts.
Log in as 'root' with no password.
Run fdisk on /dev/ubd1 to put a partition table on the new
filesystem:
host% # fdisk /dev/ubd1
o [return]
to make a new partition table
n [return]
p [return]
1 [return]
[return]
[return]
to create a
single partition spanning the whole device
w [return]
to write it out
Run 'setup'.
Answer the questions. The following answers work for me. Others
may work as well.
Remap your keyboard : 2
Continue without a swapfile : y
[a]dd more software, or [i]nstall from scratch : i
Install fresh, or add software to your current system : i
format this partition : n
From which source will you be installing Linux : 5
CD-ROM type : 1
Enter device name : /dev/ubd2
Which type of installation do you want : slakware
Which software sets do you want to install : [ up to you... ]
Do you want to use (slow) PROMPT mode : [ up to you... ]
Setup will now install everything you asked for. If you selected
prompt mode, it will ask you whether you want each optional package.
INSTALLING A LINUX KERNEL : 4
It is recommended that you make a boot disk : n
Modem Setup : n
Mouse Setup : n
Do you have a CD-ROM : n
Would you like to try out some custom screen fonts : n
LILO INSTALLATION : 7
Configuration of 'gpm' : n
Would you like to configure your timezone : [ up to you... ]
Would you like to set a root password : y
Untar /initfiles.tar:
UML#
cd /mnt
UML#
tar xpf /initfiles.tar
This is needed so that the filesystem will boot a kernel that has
devfs turned on.
UML# halt
This will hang after "INIT: no more processes left in this
runlevel". Do a ps and kill all the "linux" processes, plus the
leftover xterm.
This is a transcript of a Slackware installation following the
directions in the README provided in the Slackware installation kit.
This is fairly old, but
Martin Volf
has run the install under UML with Slackware 8.0 and put a page of
hints
here
First, the main console:
~/linux/2.3.26/um 1003: dd if=/dev/zero of=root_fs_slackware bs=$((1024*1024))\
count=100
100+0 records in
100+0 records out
~/linux/2.3.26/um 1004: linux ubd0=../distros/slackware_7.0/inst_fs_slackware_7_0 \
ubd1=root_fs_slackware ubd2r=/dev/cdrom devfs=nomount rw no-xterm
tracing thread pid = 14162
Linux version 2.4.0-test7-1um (jdike@ccure.karaya.com) (gcc version \
egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)) #1 Thu Aug 31 11:06:09 \
EST 2000
On node 0 totalpages: 4096
zone(0): 0 pages.
zone(1): 4096 pages.
zone(2): 0 pages.
Kernel command line: ubd0=../distros/slackware_7.0/inst_fs_slackware_7_0 \
ubd1=root_fs_slackware ubd2r=/dev/cdrom devfs=nomount rw no-xterm \
root=/dev/ubd0
Calibrating delay loop... 444.33 BogoMIPS
Memory: 16116k available
Dentry-cache hash table entries: 2048 (order: 2, 16384 bytes)
Buffer-cache hash table entries: 1024 (order: 0, 4096 bytes)
Page-cache hash table entries: 4096 (order: 2, 16384 bytes)
Inode-cache hash table entries: 1024 (order: 1, 8192 bytes)
VFS: Diskquotas version dquot_6.4.0 initialized
POSIX conformance testing by UNIFIX
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 1024 bind 1024)
Starting kswapd v1.7
pty: 256 Unix98 ptys configured
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: registered device at major 7
loop: enabling 8 loop devices
User-mode Linux network interface 0.005 (eth0)
User-mode Linux network interface 0.005 (eth1)
User-mode Linux network interface 0.005 (eth2)
User-mode Linux network interface 0.005 (eth3)
Initializing stdio console driver
Initializing software serial port version 0
serial line 0 assigned pty /dev/ptyp1
ssl receive thread is pid 14168
devfs: v0.102 (20000622) Richard Gooch (rgooch@atnf.csiro.au)
devfs: devfs_debug: 0x0
devfs: boot_options: 0x2
EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended
VFS: Mounted root (ext2 filesystem).
INIT: version 2.76 booting
none on /proc type proc (rw)
Virtual console #4 opened on /dev/ptyp2
INIT: Entering runlevel: 4
Then, over to one of the virtual consoles:
Welcome to the Slackware Linux installation disk! (version 7.0.0-text)
###### IMPORTANT! READ THE INFORMATION BELOW CAREFULLY. ######
- You will need one or more partitions of type 'Linux native' prepared. It is
also recommended that you create a swap partition (type 'Linux swap') prior
to installation. For more information, run 'setup' and read the help file.
- If you're having problems that you think might be related to low memory (this
is possible on machines with 8 or less megabytes of system memory), you can
try activating a swap partition before you run setup. After making a swap
partition (type 82) with cfdisk or fdisk, activate it like this:
mkswap /dev/<partition> ; swapon /dev/<partition>
- Once you have prepared the disk partitions for Linux, type 'setup' to begin
the installation process.
You may now login as 'root'.
slackware login: root
Linux 2.4.0-test7-1um.
If you're upgrading an existing Slackware system, you might want to
remove old packages before you run 'setup' to install the new ones. If
you don't, your system will still work but there might be some old files
left laying around on your drive.
Just mount your Linux partitions under /mnt and type 'pkgtool'. If you
don't know how to mount your partitions, type 'pkgtool' and it will tell
you how it's done.
To partition your hard drive(s), use 'cfdisk' or 'fdisk'.
To activate PCMCIA/Cardbus devices needed for installation, type 'pcmcia'.
To activate network devices needed for installation, type 'network'.
To start the main installation, type 'setup'.
# mke2fs /dev/ubd1
mke2fs 1.15, 18-Jul-1999 for EXT2 FS 0.5b, 95/08/09
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
25688 inodes, 102400 blocks
5120 blocks (5.00%) reserved for the super user
First data block=1
13 block groups
8192 blocks per group, 8192 fragments per group
1976 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729,
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
# fdisk /dev/ubd1
Device contains neither a valid DOS partition table, nor Sun or SGI disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 25600.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): o
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 25600.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-25600, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-25600, default 25600):
Using default value 25600
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
# setup
Welcome to Slackware Linux Setup (v. 7.0.0-text)
Linux supports many different keyboard configurations. If you are not
using a US keyboard, you will probably want to remap your keyboard.
Would you like to remap your keyboard?
1 - yes
2 - no
Your choice (1/2)? 2
You have not created a swap partition with Linux fdisk.
Do you want to continue without a swapfile ([y]es, [n]o)? y
The following partition is available for Linux:
Device Boot Start End Blocks Id System
/dev/hdb1 1 25600 52428784 83 Linux native
If this is the root partition of an existing Linux system, you
may add more software to the existing system, or you may
reformat the partition and install from scratch.
Would you like to [a]dd more software, or [i]nstall from scratch? i
Since you've chosen to install Linux from scratch, we want to be
sure you know how to proceed, and we also want to give you one last
chance to change your mind. When using this option, you must install
to a blank partition. If you have not already formatted it manually
then you must format it when prompted.
Enter [i] again to install from scratch, or [a] to add
software to your existing system.
Install fresh, or add software to your current system? ([i]nstall, [a]dd)? i
If this partition has not been formatted, you should format it.
Would you like to format this partition ([y]es, [n]o, [c]heck sectors too)? n
Mounting your new root partition:
Root partition mounted.
Installing all software on /dev/hdb1.
SOURCE MEDIA SELECTION
1 -- Install from a hard drive partition.
2 -- Install from floppy disks.
3 -- Install via NFS.
4 -- Install from a pre-mounted directory.
5 -- Install from CD-ROM.
From which source will you be installing Linux (1/2/3/4/5)? 5
INSTALLING FROM SLACKWARE CD-ROM
What type of CD-ROM drive do you have?
1 -- Works with most ATAPI/IDE CD drives (/dev/hd*)
2 -- SCSI (/dev/scd0 or /dev/scd1)
3 -- Sony CDU31A/CDU33A (/dev/sonycd)
4 -- Sony 531/535 (/dev/cdu535)
5 -- Mitsumi (proprietary interface, not IDE) (/dev/mcd)
6 -- New Mitsumi (also not IDE) (/dev/mcdx0)
7 -- Sound Blaster Pro/Panasonic (/dev/sbpcd)
8 -- Aztech/Orchid/Okano/Wearnes (/dev/aztcd)
9 -- Phillips and some ProAudioSpectrum16 (/dev/cm206cd)
10 -- Goldstar R420 (/dev/gscd)
11 -- Optics Storage 8000 (/dev/optcd)
12 -- Sanyo CDR-H94 + ISP16 soundcard (/dev/sjcd)
13 -- Try to scan for your CD drive
CD-ROM type (1/2/3/4/5/6/7/8)? 1
SELECT IDE DEVICE
Enter the device name that represents your IDE CD-ROM drive.
This will probably be one of these (in the order of most to least
likely): /dev/hdb /dev/hdc /dev/hdd /dev/hde /dev/hdf /dev/hdg
/dev/hdh /dev/hda
Enter device name: /dev/ubd2
CHOOSE YOUR INSTALLATION METHOD
With Slackware, you can run most of the system from the CD-ROM if you're
short of drive space or if you just want to test Linux without going
through a complete installation.
slakware Normal installation to hard drive
slaktest Link /usr->/cdrom/live/usr to run mostly from CD-ROM
Which type of installation do you want (slakware or slaktest)? slakware
SERIES SELECTION:
These series of packages (and possibly more) are available:
A - Base Linux system
AP - Various applications that do not need X
D - Program Development (C, C++, Kernel source, Lisp, Perl, etc.)
DES - crypt() add-on
E - GNU Emacs
F - FAQ lists
GTK - GTK+ apps and libs (including GNOME)
K - Linux kernel source
KDE - Qt and KDE (the K Desktop Environment)
N - Networking (TCP/IP, UUCP, Mail)
T - TeX
TCL - Tcl/Tk/TclX, Tcl language, and Tk toolkit for developing X apps
X - XFree86 Base X Window System
XAP - X Window Applications
XD - XFree86 X11 server development system
XV - XView (OpenLook [virtual] Window Manager, apps)
Y - Games (that do not require X)
Specify which software series you wish to install on the prompt below. For
example, to install the base system, X, and KDE, you'd enter: a kde x
To install all of the available catagories, just hit enter here.
Which software sets do you want to install? a ap
SOFTWARE INSTALLATION
Next, software packages are going to be transfered on to your hard drive.
If you use PROMPT mode, required packages are installed automatically, but
you'll be asked if you want other packages before they are installed. If you
have enough hard drive space reserved, it's much faster to NOT use PROMPT mode.
When you don't use PROMPT mode, the install program automatically installs
everything from the software series you have selected.
The package defaults are user definable -- you may set any package to be added
or skipped automatically by editing your choices into a file called TAGFILE
that will be found on the first disk of each series. There will also be a
copy of the original tagfile called TAGFILE.ORG available in case you want
to restore the default settings. The tagfile contains all the instructions
needed to completely automate your installation.
NOTE: Software may be added to your system after the initial installation.
A copy of setup will be placed in your /sbin directory. Just type 'setup' to
add software to your system. Another script, PKGTOOL, may be used to add
software packages from the current directory, or to cleanly remove packages
that you want uninstalled.
Do you want to use (slow) PROMPT mode (y/n)? n
PROMPT mode not selected.
Installing all the packages in each selected series.
--- Installing disk series ==>a<==
Auto-installing package ==>aaa_base<== Priority: [required]
aaa_base: Basic Linux filesystem package.
aaa_base:
aaa_base: Sets up the empty directory tree for Slackware and adds an email to
aaa_base: root's mailbox welcoming them to Linux. :) This package should be
aaa_base: installed first, and never uninstalled.
aaa_base:
aaa_base:
Size: Compressed: 6K, uncompressed: 90K.
Installing package aaa_base...
Executing installation script for package aaa_base...
Done installing package aaa_base.
Auto-installing package ==>aoutlibs<== Priority: [required]
aoutlibs: a.out (libc4) shared libraries
aoutlibs:
aoutlibs: These shared libraries provide support for running Linux binaries
aoutlibs: compiled in the now obsolete a.out binary format. These libraries
aoutlibs: can be found in /usr/i386-slackware-linux-gnuaout/lib.
aoutlibs:
aoutlibs: Adds crt0.o, libvga.so.1.2.9, libdb.so.1.85.1, libc.so.4.7.6,
aoutlibs: libcurses.so.0.1.2, libm.so.4.6.27.
aoutlibs:
Size: Compressed: 311K, uncompressed: 1030K.
Installing package aoutlibs...
Executing installation script for package aoutlibs...
Done installing package aoutlibs.
Auto-installing package ==>bash<== Priority: [required]
bash: GNU bash-2.03
bash:
bash: The GNU Bourne-Again SHell. Bash is a sh-compatible command
bash: interpreter that executes commands read from the standard input or
bash: from a file. Bash also incorporates useful features from the Korn
bash: and C shells (ksh and csh). Bash is ultimately intended to be a
bash: conformant implementation of the IEEE Posix Shell and Tools
bash: specification (IEEE Working Group 1003.2).
bash:
bash: Bash must be present for the system to boot properly.
bash:
Size: Compressed: 380K, uncompressed: 680K.
Installing package bash...
Executing installation script for package bash...
Done installing package bash.
Auto-installing package ==>bash1<== Priority: [recommended]
bash1: GNU bash-1.14.7
bash1:
bash1: The GNU Bourne-Again SHell. Bash is a sh-compatible command language
bash1: interpreter that executes commands read from the standard input or
bash1: from a file. Bash also incorporates useful features from the Korn
bash1: and C shells (ksh and csh). Bash is ultimately intended to be a
bash1: conformant implementation of the IEEE Posix Shell and Tools
bash1: specification (IEEE Working Group 1003.2).
bash1:
bash1: Some older scripts may need /bin/bash1 instead of /bin/bash.
bash1:
Size: Compressed: 238K, uncompressed: 450K.
Installing package bash1...
Executing installation script for package bash1...
Done installing package bash1.
Auto-installing package ==>bzip2<== Priority: [required]
bzip2: bzip2 version 0.9.0c (a block-sorting file compressor)
bzip2:
bzip2: Bzip2 compresses files using the Burrows-Wheeler block sorting text
bzip2: compression algorithm, and Huffman coding. Compression is generally
bzip2: considerably better than that achieved by more conventional
bzip2: LZ77/LZ78-based compressors, and approaches the performance of the PPM
bzip2: family of statistical compressors.
bzip2:
bzip2: Julian Seward <jseward@acm.org> is the author of bzip2.
bzip2:
Size: Compressed: 87K, uncompressed: 260K.
Installing package bzip2...
Executing installation script for package bzip2...
Done installing package bzip2.
Auto-installing package ==>cpio<== Priority: [required]
cpio: The GNU cpio backup and archiving utility v. 2.4.2
cpio:
cpio: This is GNU cpio, a program to manage archives of files. This package
cpio: also includes mt, a tape drive control program. cpio copies files into
cpio: or out of a cpio or tar archive, which is a file that contains other
cpio: files plus information about them, such as their pathname, owner,
cpio: timestamps, and access permissions. The archive can be another file on
cpio: the disk, a magnetic tape, or a pipe.
cpio:
Size: Compressed: 72K, uncompressed: 200K.
Installing package cpio...
Executing installation script for package cpio...
Done installing package cpio.
Auto-installing package ==>devs<== Priority: [required]
devs: Device files
devs:
devs: This package creates special files in the /dev directory that
devs: represent your system's hardware.
devs:
Size: Compressed: 57K, uncompressed: 1950K.
Installing package devs...
Executing installation script for package devs...
Done installing package devs.
Auto-installing package ==>find<== Priority: [required]
find: GNU findutils-4.1
find:
find: This package contains the GNU find, xargs, and locate programs.
find: find and xargs comply with POSIX 1003.2. They also support some
find: additional options, some borrowed from Unix and some unique to GNU.
find:
Size: Compressed: 80K, uncompressed: 150K.
Installing package find...
Done installing package find.
Auto-installing package ==>getty<== Priority: [optional]
getty: getty_ps 2.0.7j
getty:
getty: /sbin/getty and /sbin/uugetty.
getty: These control the process of logging into your system, and may be used
getty: instead of 'agetty', which is the default getty included with the
getty: util-linux package. getty_ps supports a number of enhancements such as
getty: ringback support.
getty:
Size: Compressed: 59K, uncompressed: 150K.
Installing package getty...
Done installing package getty.
Auto-installing package ==>bin<== Priority: [required]
bin: Binaries that go in /bin and /usr/bin.
bin:
bin: Various required utilities and programs, such as:
bin: apmd_3.0beta8-1, asapm-2.2, at-3.1.7, banners, bpe, compress,
bin: dcron-2.3.3, ed-0.2, eject_1.5-5, file_3.27-4, gawk-3.0.4,
bin: hdparm-3.5, indent-2.2.0, lha-1.00, makewhatis, mkdosfs-0.4,
bin: patch-2.5.3, rpm2targz, sed-3.02, sharutils-4.2c, splitvt-1.6.3,
bin: time-1.7, todos, unarj230, zoo-2.10.
bin:
Size: Compressed: 971K, uncompressed: 1960K.
Installing package bin...
Executing installation script for package bin...
Done installing package bin.
Auto-installing package ==>e2fsprog<== Priority: [required]
e2fsprog: e2fsprogs-1.15
e2fsprog:
e2fsprog: Utilities needed to create and maintain ext2 filesystems under Linux.
e2fsprog: Included in this package are: chattr, lsattr, mke2fs, mklost+found,
e2fsprog: tune2fs, fsck, e2fsck, e2label, debugfs, dumpe2fs, and badblocks.
e2fsprog:
e2fsprog: These utilities were written by Remy Card (the developer and
e2fsprog: maintainer of the ext2 fs) and Theodore T'so.
e2fsprog:
Size: Compressed: 234K, uncompressed: 650K.
Installing package e2fsprog...
Executing installation script for package e2fsprog...
Done installing package e2fsprog.
Auto-installing package ==>gzip<== Priority: [required]
gzip: GNU zip compression utilities. (v. 1.2.4a)
gzip:
gzip: Gzip reduces the size of the named files using Lempel-Ziv coding
gzip: (LZ77). Whenever possible, each file is replaced by one with the
gzip: extension .gz, while keeping the same ownership modes, access and
gzip: modification times.
gzip:
Size: Compressed: 51K, uncompressed: 100K.
Installing package gzip...
Executing installation script for package gzip...
Done installing package gzip.
Auto-installing package ==>less<== Priority: [required]
less: less-340
less:
less: Less is a paginator similar to more (1), but which allows backward
less: movement in the file as well as forward movement. Also, less does not
less: have to read the entire input file before starting, so with large
less: input files it starts up faster than text editors like vi (1).
less:
less: Less was written by Mark Nudelman.
less:
Size: Compressed: 67K, uncompressed: 140K.
Installing package less...
Done installing package less.
Auto-installing package ==>cxxlibs<== Priority: [required]
cxxlibs: C++ shared libraries
cxxlibs:
cxxlibs: This package contains the shared libraries needed to run dynamically
cxxlibs: linked C++ binaries.
cxxlibs:
cxxlibs: Includes libstdc++.so.2.7.2.8,
cxxlibs: libstdc++.so.2.8.0,
cxxlibs: libstdc++.so.2.9.0,
cxxlibs: libstdc++-2-libc6.1-1-2.9.0.so.
cxxlibs:
Size: Compressed: 392K, uncompressed: 1250K.
Installing package cxxlibs...
Executing installation script for package cxxlibs...
Done installing package cxxlibs.
Auto-installing package ==>elflibs<== Priority: [required]
elflibs: Assorted ELF shared libraries
elflibs:
elflibs: This is a collection of shared libraries needed to run Linux code in
elflibs: the ELF binary format. ELF (Executable and Linking Format) is the
elflibs: standard Linux binary format. These libraries are gathered from
elflibs: packages outside of the A series, and are intended to give a fairly
elflibs: complete set of runtime ELF libraries when added to the glibcso
elflibs: package (also in the A series). If you're doing a full installation,
elflibs: these libraries are redundant, but installing them doesn't hurt.
elflibs:
Size: Compressed: 734K, uncompressed: 1790K.
Installing package elflibs...
Executing installation script for package elflibs...
Done installing package elflibs.
Auto-installing package ==>gpm<== Priority: [recommended]
gpm: gpm-1.17.8
gpm:
gpm: The general purpose mouse server, or gpm, allows you to use the mouse
gpm: to cut and paste text from the screen. It also acts as a mouse
gpm: server for applications running on the Linux console, such as the
gpm: Midnight Commander file manager and dialog.
gpm:
gpm: NOTE: This program may cause problems when you start X on systems
gpm: that do not use a serial mouse. If you get an 'unable to open mouse
gpm: device' error from X, disable /etc/rc.d/rc.gpm.
gpm:
Size: Compressed: 196K, uncompressed: 500K.
Installing package gpm...
Executing installation script for package gpm...
Done installing package gpm.
Auto-installing package ==>elvis<== Priority: [required]
elvis: elvis-2.1_4
elvis:
elvis: Elvis is a text editor. It is intended to be a modern replacement
elvis: for the classic ex/vi editor of UNIX fame. Elvis supports many new
elvis: features, including multiple edit buffers, multiple windows, and a
elvis: variety of display modes.
elvis:
elvis: As elvis is Slackware's default vi, this is a required package.
elvis:
Size: Compressed: 472K, uncompressed: 1330K.
Installing package elvis...
Executing installation script for package elvis...
Done installing package elvis.
Auto-installing package ==>etc<== Priority: [required]
etc: /etc configuration files
etc:
etc: System configuration files. The /etc directory is traditionally the
etc: location where configuration files are found.
etc:
Size: Compressed: 212K, uncompressed: 670K.
Installing package etc...
Executing installation script for package etc...
Done installing package etc.
Auto-installing package ==>fileutls<== Priority: [required]
fileutls: fileutils-4.0
fileutls:
fileutls: These are the GNU file management utilities. Most of these programs
fileutls: have significant advantages over their Unix counterparts, such as
fileutls: greater speed, additional options, and fewer arbitrary limits.
fileutls:
fileutls: These utilities are included:
fileutls: chgrp chmod chown cp dd df dircolors du ginstall ln ls
fileutls: mkdir mkfifo mknod mv rm rmdir sync touch
fileutls:
Size: Compressed: 404K, uncompressed: 1180K.
Installing package fileutls...
Executing installation script for package fileutls...
Done installing package fileutls.
Auto-installing package ==>grep<== Priority: [required]
grep: GNU grep 2.3
grep:
grep: Grep searches the named input files (or standard input if no files are
grep: named, or the file name - is given) for lines containing a match to
grep: the given pattern. This is the 'fastest grep in the west' (we hope).
grep:
Size: Compressed: 131K, uncompressed: 340K.
Installing package grep...
Executing installation script for package grep...
Done installing package grep.
Auto-installing package ==>isapnp<== Priority: [optional]
isapnp: isapnptools-1.18
isapnp:
isapnp: These programs allow ISA Plug-And-Play devices to be
isapnp: configured on a Linux machine. See the documentation in
isapnp: /usr/doc/isapnptools-1.18.
isapnp:
Size: Compressed: 89K, uncompressed: 260K.
Installing package isapnp...
Done installing package isapnp.
Auto-installing package ==>floppy<== Priority: [required]
floppy: floppy disk utilities
floppy:
floppy: Contains fdutils-5.3, tools to test and format floppy disks; and
floppy: mtools-3.9.6, a collection of utilities for using DOS/Windows floppy
floppy: disks with Linux.
floppy:
Size: Compressed: 365K, uncompressed: 680K.
Installing package floppy...
Executing installation script for package floppy...
Done installing package floppy.
Auto-installing package ==>fsmods<== Priority: [required]
fsmods: Filesystem modules for Linux 2.2.13
fsmods:
fsmods: A kernel module is a piece of object code that can be dynamically
fsmods: loaded into the Linux kernel to provide new kernel functions. These
fsmods: modules support additional filesystems, such as HPFS, HFS, and NTFS.
fsmods:
Size: Compressed: 497K, uncompressed: 1330K.
Installing package fsmods...
Done installing package fsmods.
Auto-installing package ==>hdsetup<== Priority: [required]
hdsetup: The Slackware setup/package maintenance system v. 7.0.0
hdsetup:
hdsetup: The full screen and tty versions of setup and pkgtool, two utilities
hdsetup: included with Slackware that allow you to easily install software
hdsetup: packages, remove software packages, and review the installed software
hdsetup: packages.
hdsetup:
hdsetup: Also contains the command line utilities 'installpkg', 'removepkg',
hdsetup: 'makepkg', 'explodepkg', and 'upgradepkg' that install, remove, build,
hdsetup: examine, and upgrade software packages. Man pages are included.
hdsetup:
Size: Compressed: 138K, uncompressed: 700K.
Installing package hdsetup...
Executing installation script for package hdsetup...
Done installing package hdsetup.
Auto-installing package ==>infozip<== Priority: [required]
infozip: Info-ZIP's zip 2.2 and unzip 5.40 utilities
infozip:
infozip: zip is a compression and file packaging utility for Unix, VMS, MSDOS,
infozip: OS/2, Windows NT, Minix, Atari and Macintosh, Amiga and Acorn RISC OS.
infozip: It is analogous to a combination of the UNIX commands tar(1) and
infozip: compress(1) and is compatible with PKZIP (Phil Katz's ZIP). A
infozip: companion program (unzip(1L)), unpacks zip archives.
infozip:
infozip:
infozip:
Size: Compressed: 227K, uncompressed: 540K.
Installing package infozip...
Executing installation script for package infozip...
Done installing package infozip.
Auto-installing package ==>loadlin<== Priority: [recommended]
loadlin: LOADLIN v1.6a
loadlin:
loadlin: LOADLIN.EXE, a loader (running under DOS) for LINUX kernel images
loadlin: fully supporting the kernel's command line feature.
loadlin:
loadlin: LOADLIN is the safest way to boot Linux from your hard disk, if you
loadlin: have a bootable DOS partition on your machine.
loadlin:
loadlin: LOADLIN is (C) 1994...1995 Hans Lermen (lermen@elserv.ffm.fgan.de)
loadlin:
Size: Compressed: 93K, uncompressed: 100K.
Installing package loadlin...
Done installing package loadlin.
Auto-installing package ==>glibcso<== Priority: [required]
glibcso: glibc-2.1.2 runtime support
glibcso:
glibcso: This package contains the shared libraries, binaries, and support
glibcso: files required to run most Linux applications linked with glibc.
glibcso:
Size: Compressed: 749K, uncompressed: 1920K.
Installing package glibcso...
Executing installation script for package glibcso...
Done installing package glibcso.
Auto-installing package ==>ibcs2<== Priority: [optional]
ibcs2: Intel Binary Compatibility Specification module
ibcs2:
ibcs2: The iBCS module allows many binaries from other OS's to run on Linux:
ibcs2:
ibcs2: i386 BSD (386BSD, FreeBSD, NetBSD, BSDI/386) - very alpha.
ibcs2: SVR4 (Interactive, Unixware, USL, Dell etc.)
ibcs2: SVR3 generic
ibcs2: SCO (SVR3 with extensions for symlinks and long filenames)
ibcs2: Wyse V/386 (SVR3 with extensions for symlinks)
ibcs2: Xenix V/386 (386 small model binaries only)
ibcs2:
Size: Compressed: 427K, uncompressed: 1290K.
Installing package ibcs2...
Executing installation script for package ibcs2...
Done installing package ibcs2.
Auto-installing package ==>lpr<== Priority: [recommended]
lpr: lpr-0.35-6
lpr:
lpr: This is the lpr printing suite ported from BSD. It's the most
lpr: standard UN*X printing system in current use. If you're interested
lpr: in automatic detection of PostScript and DVI files, check out the
lpr: apsfilter add-on in Slackware's AP package series.
lpr:
Size: Compressed: 74K, uncompressed: 180K.
Installing package lpr...
Executing installation script for package lpr...
Done installing package lpr.
Auto-installing package ==>pciutils<== Priority: [optional]
pciutils: pciutils-2.0 (Linux PCI utilities)
pciutils:
pciutils: lspci displays detailed information about all PCI buses and devices
pciutils: in the system, replacing the original /proc/pci interface.
pciutils:
pciutils: setpci allows reading from and writing to PCI device configuration
pciutils: registers. For example, you can adjust the latency timers with it.
pciutils:
pciutils: See manual pages for more details.
pciutils:
Size: Compressed: 50K, uncompressed: 130K.
Installing package pciutils...
Done installing package pciutils.
Auto-installing package ==>umsprogs<== Priority: [required]
umsprogs: umsdos-0.9
umsprogs:
umsprogs: Various utilities needed by Jacques Gelinas' UMSDOS filesystem.
umsprogs:
Size: Compressed: 21K, uncompressed: 60K.
Installing package umsprogs...
Executing installation script for package umsprogs...
Done installing package umsprogs.
Auto-installing package ==>ide<== Priority: [recommended]
ide: Linux kernel version 2.2.13, without SCSI support
ide:
ide: A Linux kernel for computers that do not need SCSI support. You MUST
ide: install a kernel image in order for your system to boot. This kernel
ide: supports IDE hard drives and IDE CD-ROM drives. Other drivers (such
ide: as for CD-ROM drives on proprietary interfaces, or ethernet cards)
ide: may be loaded as modules. See /etc/rc.d/rc.modules for examples.
ide:
Size: Compressed: 792K, uncompressed: 990K.
Installing package ide...
Executing installation script for package ide...
Done installing package ide.
Auto-installing package ==>ldso<== Priority: [required]
ldso: ld.so 1.9.9, the dynamic linker/loader
ldso:
ldso: ld.so completes the final process of linking all necessary references
ldso: to sharable objects and unreferenced symbols in an impure executable,
ldso: (usually a dynamically linked executable), to produce a runnable file.
ldso: Nearly all Linux binaries are 'incomplete' and require further linking
ldso: at run time. ld.so's job is thus to complete the linking process
ldso: started at compilation. ld.so was written by David Engel,
ldso: Eric Youngdale, Peter MacDonald, Hongjiu Lu, Linus Torvalds,
ldso: Lars Wirzenius and Mitch D'Souza.
ldso:
Size: Compressed: 195K, uncompressed: 470K.
Installing package ldso...
Executing installation script for package ldso...
Done installing package ldso.
Auto-installing package ==>lilo<== Priority: [required]
lilo: LILO 21
lilo:
lilo: Generic Boot Loader for Linux ('LInux LOader') by Werner Almesberger.
lilo: LILO boots Linux from your hard drive. It can also boot other
lilo: operating systems such as MS-DOS and OS/2, and can even boot DOS from
lilo: the second hard drive. LILO comes with utilities and documentation
lilo: that make it easier to install, such as 'liloconfig' and 'QuickStart'.
lilo: NOTE: Installing boot loaders is inherently dangerous. Be sure to have
lilo: some means to boot your system from a different media if you install
lilo: LILO on your hard disk.
lilo:
Size: Compressed: 153K, uncompressed: 530K.
Installing package lilo...
Done installing package lilo.
Auto-installing package ==>man<== Priority: [required]
man: man-1.5g
man:
man: The man package is a collection of tools used for searching and
man: reading the online system documentation. In fact, on most UNIX-like
man: operating systems it is the primary means of finding out how programs
man: on the system work. For example, 'man man' will display the
man: documentation for man itself.
man:
man: Man requires the groff text processing package.
man:
Size: Compressed: 117K, uncompressed: 290K.
Installing package man...
Done installing package man.
Auto-installing package ==>kbd<== Priority: [recommended]
kbd: kbd-0.99
kbd:
kbd: Load and save keyboard mappings. Needed if you are not using the US
kbd: keyboard map. This package also contains utilities to change your
kbd: console fonts - if you install it you'll get a menu later on that lets
kbd: you select from many different fonts. If you like one, you can make it
kbd: your default font. A new default font can be chosen at any time by
kbd: typing 'fontconfig'. This package also includes fonts from the
kbd: kbd_fonts.tar.gz package uploaded to metalab.
kbd:
Size: Compressed: 763K, uncompressed: 1760K.
Installing package kbd...
Executing installation script for package kbd...
Done installing package kbd.
Auto-installing package ==>minicom<== Priority: [recommended]
minicom: Minicom 1.82-3
minicom:
minicom: Minicom - a full featured menu-driven communications package similar
minicom: to the DOS program 'Telix'. Also includes sz/rz - utilities used to
minicom: upload and download files using the Zmodem protocol.
minicom:
minicom:
Size: Compressed: 122K, uncompressed: 290K.
Installing package minicom...
Executing installation script for package minicom...
Done installing package minicom.
Auto-installing package ==>modutils<== Priority: [required]
modutils: modutils-2.1.121
modutils:
modutils: Utilities to use kernel modules. These tools are used for loading
modutils: and unloading chunks of kernel code on the fly, to add support for a
modutils: CD-ROM drive or ethernet card, for instance.
modutils:
modutils: Many Linux drivers are provided as kernel modules, and some packages
modutils: (like the PCMCIA package) require the module utilities to work, so you
modutils: probably want to install these.
modutils:
Size: Compressed: 113K, uncompressed: 290K.
Installing package modutils...
Executing installation script for package modutils...
Done installing package modutils.
Auto-installing package ==>procps<== Priority: [required]
procps: procps-2.0.2, psmisc-18, procinfo-16
procps:
procps: Utilities for displaying process and memory information written by
procps: Michael K. Johnson, Werner Almesberger, Charles Blake, Branko
procps: Lankester, Robert Nation, David Engel, Larry Greenfield, Roger Binns,
procps: Sander van Malssen, and Brian Edmonds.
procps:
Size: Compressed: 209K, uncompressed: 560K.
Installing package procps...
Executing installation script for package procps...
Done installing package procps.
Auto-installing package ==>sysklogd<== Priority: [required]
sysklogd: Sysklogd 1.3-33
sysklogd:
sysklogd: Dr. Greg Wettstein and Stephen Tweedie's syslogd/klogd.
sysklogd:
sysklogd: This package contains a modified version of syslogd for the Linux
sysklogd: environment. An additional utility, klogd, is included which allows
sysklogd: kernel logging to be directed through the syslogd facility.
sysklogd: Syslogd and klogd are started when your system boots.
sysklogd:
Size: Compressed: 57K, uncompressed: 130K.
Installing package sysklogd...
Executing installation script for package sysklogd...
Done installing package sysklogd.
Auto-installing package ==>libc5<== Priority: [required]
libc5: Linux libc5 ELF shared libraries
libc5:
libc5: These libraries, found in /usr/i386-slackware-linux-gnulibc1/lib, are
libc5: needed to run pre-glibc Linux ELF binaries.
libc5:
Size: Compressed: 958K, uncompressed: 2580K.
Installing package libc5...
Executing installation script for package libc5...
Done installing package libc5.
Auto-installing package ==>sh_utils<== Priority: [required]
sh_utils: GNU sh-utils-1.16
sh_utils:
sh_utils: This is a package of small shell programming utilities. They are
sh_utils: mostly compliant with POSIX.2, where applicable.
sh_utils:
sh_utils: The programs in this package are:
sh_utils: basename chroot date dirname echo env expr false logname nice
sh_utils: nohup pathchk printenv printf sleep stty tee test true tty
sh_utils: uname who whoami yes
sh_utils:
Size: Compressed: 224K, uncompressed: 620K.
Installing package sh_utils...
Executing installation script for package sh_utils...
Done installing package sh_utils.
Auto-installing package ==>modules<== Priority: [required]
modules: Linux kernel modules for 2.2.13
modules:
modules: A kernel module is a piece of object code that can be dynamically
modules: loaded into the Linux kernel to provide new kernel functions. Most
modules: of these modules provide support for devices such as CD-ROM drives,
modules: tape drives, and ethernet cards. You can choose which modules to load
modules: by editing /etc/rc.d/rc.modules.
modules:
Size: Compressed: 1177K, uncompressed: 3110K.
Installing package modules...
Executing installation script for package modules...
Done installing package modules.
Auto-installing package ==>pcmcia<== Priority: [recommended]
pcmcia: pcmcia-cs-3.0.14
pcmcia:
pcmcia: Card Services for Linux is a complete PCMCIA support package. It
pcmcia: includes a set of loadable kernel modules that implement a version of
pcmcia: the PCMCIA Card Services applications program interface, a set of
pcmcia: client drivers for specific cards, and a card manager daemon that can
pcmcia: respond to card insertion and removal events, loading and unloading
pcmcia: drivers on demand. It supports ``hot swapping'' of PCMCIA cards, so
pcmcia: cards can be inserted and ejected at any time.
pcmcia:
Size: Compressed: 489K, uncompressed: 1320K.
Installing package pcmcia...
Done installing package pcmcia.
Auto-installing package ==>shadow<== Priority: [required]
shadow: Shadow password suite (shadow-19990607)
shadow:
shadow: This set of login related programs utilizes an alternate, non-readable
shadow: file to contain the actual encrypted passwords. This is presumed to
shadow: increase system security by increasing the difficulty with which
shadow: system crackers obtain encrypted passwords. It was written by
shadow: Julianne Frances Haugh and the Linux port is maintained by Marek
shadow: Michalkiewicz.
shadow:
Size: Compressed: 338K, uncompressed: 930K.
Installing package shadow...
Executing installation script for package shadow...
Done installing package shadow.
Auto-installing package ==>sysvinit<== Priority: [required]
sysvinit: sysvinit-2.76-4
sysvinit:
sysvinit: System V style init programs by Miquel van Smoorenburg that control
sysvinit: the booting and shutdown of your system. These support a number of
sysvinit: system runlevels, each with a specific set of utilities spawned. For
sysvinit: example, the normal system runlevel is 3, which starts agetty on
sysvinit: virtual consoles tty1 - tty6. Runlevel 4 starts xdm. Runlevel 0 shuts
sysvinit: the system down. See the documentation in /usr/doc/sysvinit and the
sysvinit: scripts in /etc/rc.d for more information.
sysvinit:
Size: Compressed: 218K, uncompressed: 520K.
Installing package sysvinit...
Executing installation script for package sysvinit...
Done installing package sysvinit.
Auto-installing package ==>tcsh<== Priority: [recommended]
tcsh: tcsh 6.08
tcsh:
tcsh: tcsh is an enhanced but completely compatible version of the Berkeley
tcsh: UNIX C shell, csh(1). It is a command language interpreter usable both
tcsh: as an interactive login shell and a shell script command processor. It
tcsh: includes a command-line editor, programmable word completion, spelling
tcsh: correction, a history mechanism, job control, and a C-like syntax.
tcsh:
Size: Compressed: 249K, uncompressed: 480K.
Installing package tcsh...
Executing installation script for package tcsh...
Done installing package tcsh.
Auto-installing package ==>scsi<== Priority: [recommended]
scsi: Linux kernel version 2.2.13, with SCSI support
scsi:
scsi: A Linux kernel for computers with SCSI and/or IDE. You MUST install a
scsi: kernel image in order for your system to boot. This kernel supports
scsi: IDE and SCSI hard drives and CD-ROM drives. Other drivers (such as
scsi: for CD-ROM drives on proprietary interfaces, or ethernet cards) may
scsi: be loaded as modules. See /etc/rc.d/rc.modules for examples.
scsi:
Size: Compressed: 1082K, uncompressed: 1300K.
Installing package scsi...
Executing installation script for package scsi...
Done installing package scsi.
Auto-installing package ==>txtutils<== Priority: [required]
txtutils: GNU textutils-1.22
txtutils:
txtutils: These are the GNU text file (actually, file contents) processing
txtutils: utilities. Most of these programs have significant advantages over
txtutils: their Unix counterparts, such as greater speed, additional options,
txtutils: and fewer arbitrary limits.
txtutils:
txtutils: The programs in this package are: cat, cksum, comm, csplit, cut,
txtutils: expand, fold, head, join, md5sum, nl, od, paste, pr, sort, split,
txtutils: sum, tac, tail, tr, unexpand, uniq, and wc.
txtutils:
Size: Compressed: 180K, uncompressed: 410K.
Installing package txtutils...
Executing installation script for package txtutils...
Done installing package txtutils.
Auto-installing package ==>scsimods<== Priority: [required]
scsimods: Linux SCSI, RAID, and CD-ROM kernel modules for Linux 2.2.13
scsimods:
scsimods: A kernel module is a piece of object code that can be dynamically
scsimods: loaded into the Linux kernel to provide new kernel functions. These
scsimods: modules support SCSI interface cards, RAID (both IDE and SCSI), and
scsimods: non-SCSI CD-ROM drives (like the kind that attach to sound cards).
scsimods:
Size: Compressed: 945K, uncompressed: 2190K.
Installing package scsimods...
Done installing package scsimods.
Auto-installing package ==>tar<== Priority: [required]
tar: GNU tar 1.13
tar:
tar: Tape ARchiver - adds and extracts files and directories to/from tape
tar: or archive files. This version of tar also supports archive
tar: compression using GNU zip or bzip2.
tar:
Size: Compressed: 295K, uncompressed: 650K.
Installing package tar...
Executing installation script for package tar...
Done installing package tar.
Auto-installing package ==>util<== Priority: [required]
util: util-linux 2.9v - A huge collection of essential utilities
util:
util: agetty arch banner cal cfdisk clear clock col colcrt colrm column
util: ctrlaltdel cytune ddate dmesg fdformat fdisk fsck.minix getopt hexdump
util: hostname hwclock ipcrm ipcs jaztool kbdrate last logger look
util: losetup mcookie mesg mkfs mkfs.minix mkswap more mount namei rdev
util: readprofile renice reset rev script setfdprm setserial setsid setterm
util: sfdisk sln strings swapon tsort tunelp ul umount update wall whereis
util: write ziptool
util:
Size: Compressed: 751K, uncompressed: 1820K.
Installing package util...
Executing installation script for package util...
Done installing package util.
Auto-installing package ==>zoneinfo<== Priority: [required]
zoneinfo: time zone database
zoneinfo:
zoneinfo: This package allows you to configure your time zone.
zoneinfo:
zoneinfo: This database comes from glibc-2.1.2, and is based on the tzdata1999e
zoneinfo: package by Arthur David Olson et.al. The latest version and more
zoneinfo: information may be found at ftp://elsie.nci.nih.gov/pub/
zoneinfo:
zoneinfo: Use the timeconfig utility to set your local time zone.
zoneinfo:
Size: Compressed: 234K, uncompressed: 1730K.
Installing package zoneinfo...
Executing installation script for package zoneinfo...
Done installing package zoneinfo.
--- Installing disk series ==>ap<==
Auto-installing package ==>apsfilt<== Priority: [recommended]
apsfilt: apsfilter-5.0.1.
apsfilt:
apsfilt: Apsfilter is a printer input filter. With apsfilter installed and
apsfilt: configured, you can print many different file types (like ASCII,
apsfilt: PostScript, and DVI, as well as compressed versions of all of these)
apsfilt: directly with 'lpr', and apsfilter will automatically handle the
apsfilt: conversion process on the fly.
apsfilt:
apsfilt: To configure apsfilter, run the SETUP program in /usr/lib/apsfilter.
apsfilt:
Size: Compressed: 186K, uncompressed: 600K.
Installing package apsfilt...
Done installing package apsfilt.
Auto-installing package ==>ash<== Priority: [optional]
ash: Kenneth Almquist's ash shell.
ash:
ash: A lightweight (62K) Bourne compatible shell. Great for machines with
ash: low memory, but does not provide all the extras of shells like bash,
ash: tcsh, and zsh. Runs most shell scripts compatible with the Bourne
ash: shell. Note that under Linux, most scripts seem to use at least some
ash: bash-specific syntax. The Slackware setup scripts are a notable
ash: exception, since ash is the shell used on the install disks. NetBSD
ash: uses ash as its /bin/sh.
ash:
Size: Compressed: 51K, uncompressed: 100K.
Installing package ash...
Executing installation script for package ash...
Done installing package ash.
Auto-installing package ==>bc<== Priority: [optional]
bc: GNU bc 1.05a - An arbitrary precision calculator language.
bc:
bc: bc is a language that supports arbitrary precision numbers with
bc: interactive execution of statements. There are some similarities in
bc: the syntax to the C programming language.
bc:
Size: Compressed: 89K, uncompressed: 210K.
Installing package bc...
Done installing package bc.
Auto-installing package ==>cdutils<== Priority: [optional]
cdutils: Tools for mastering and writing compact discs.
cdutils:
cdutils: Contains these utilities:
cdutils:
cdutils: cdwrite -- burn discs in some CD-R drives from Philips/IMS/Kodak,
cdutils: Yamaha, and HP.
cdutils: cdrecord -- burn discs in most ATAPI and SCSI CD-R drives.
cdutils: mkisofs -- create ISO9660 filesystem images.
cdutils: mkhybrid -- create hybrid ISO9660/HFS/Joliet CD-ROM images.
cdutils:
Size: Compressed: 465K, uncompressed: 1170K.
Installing package cdutils...
Done installing package cdutils.
Auto-installing package ==>diff<== Priority: [recommended]
diff: GNU diffutils-2.7
diff:
diff: The GNU diff utilities - finds differences between files. (to make
diff: source code patches, for instance)
diff:
Size: Compressed: 110K, uncompressed: 200K.
Installing package diff...
Done installing package diff.
Auto-installing package ==>enscript<== Priority: [optional]
enscript: GNU enscript 1.6.1
enscript:
enscript: Enscript converts text files to PostScript and spools the generated
enscript: output to a printer or saves it to a file. If no input files are
enscript: given, enscript processes standard input. Enscript can be extended to
enscript: handle different output media and it has many options for customizing
enscript: printouts.
enscript:
Size: Compressed: 334K, uncompressed: 1430K.
Installing package enscript...
Done installing package enscript.
Auto-installing package ==>ghostscr<== Priority: [recommended]
ghostscr: Ghostscript version 5.10
ghostscr:
ghostscr: Preview, print and process PostScript documents on both PostScript and
ghostscr: non-PostScript devices. If you need X11 support, you should also
ghostscr: install the package 'gs_x11' found on the XAP series. It contains a
ghostscr: /usr/bin/gs with support for X11.
ghostscr:
Size: Compressed: 1195K, uncompressed: 3530K.
Installing package ghostscr...
Executing installation script for package ghostscr...
Done installing package ghostscr.
Auto-installing package ==>groff<== Priority: [required]
groff: GNU troff 1.11a document formatting system.
groff:
groff: The groff package provides versions of troff, nroff, eqn, tbl, and
groff: other Unix text-formatting utilities. Groff is used to 'compile' man
groff: pages stored in groff/nroff format into a form which can be printed or
groff: displayed on the screen. These man pages are stored in compressed
groff: form in the /usr/man/man? directories.
groff:
Size: Compressed: 992K, uncompressed: 3200K.
Installing package groff...
Executing installation script for package groff...
Done installing package groff.
Auto-installing package ==>gsfonts<== Priority: [recommended]
gsfonts: Fonts for the Ghostscript interpreter/previewer.
gsfonts:
gsfonts: These fonts come from ghostscript-fonts-other-5.10.tar.gz and
gsfonts: ghostscript-fonts-std-5.10.tar.gz.
gsfonts:
Size: Compressed: 2193K, uncompressed: 3810K.
Installing package gsfonts...
Done installing package gsfonts.
Auto-installing package ==>ispell<== Priority: [optional]
ispell: ispell-3.1.20
ispell:
ispell: Ispell is a fast screen-oriented spelling checker that shows you your
ispell: errors in the context of the original file, and suggests possible
ispell: corrections when it can figure them out. Compared to UNIX spell, it
ispell: is faster and much easier to use. Ispell can also handle languages
ispell: other than English. Ispell has a long history, and many people have
ispell: contributed to the current version -- some of the major contributors
ispell: include R. E. Gorin, Pace Willisson, Walt Buehring, and Geoff
ispell: Kuenning.
ispell:
Size: Compressed: 417K, uncompressed: 1030K.
Installing package ispell...
Executing installation script for package ispell...
Done installing package ispell.
Auto-installing package ==>jed<== Priority: [optional]
jed: John E. Davis's JED 0.98-7 editor.
jed:
jed: JED is an extremely powerful but small emacs programmer's editor that
jed: is extensible in a C-like macro language and is able to perform color
jed: syntax highlighting. Among the many features:
jed: Emacs, wordstar, EDT emulation;
jed: C, Fortran, TeX, text editing modes; Full undo;
jed: GNU Emacs compatible info reader, and lots more.
jed:
jed: 8 bit clean, so you can even edit binary files!
jed:
Size: Compressed: 430K, uncompressed: 1470K.
Installing package jed...
Executing installation script for package jed...
Done installing package jed.
Auto-installing package ==>joe<== Priority: [optional]
joe: Joe text editor v2.8
joe:
joe: Joseph H. Allen's easy to use text editor, similar to WordStar[tm].
joe:
Size: Compressed: 122K, uncompressed: 300K.
Installing package joe...
Executing installation script for package joe...
Done installing package joe.
Auto-installing package ==>jove<== Priority: [optional]
jove: Jonathan's Own Version of Emacs (4.14.10)
jove:
jove: A mostly Emacs compatible text editor, with man pages and the
jove: 'teachjove' tutorial program. If you'd rather have the real thing,
jove: the actual Emacs editor/environment is available in series 'E'.
jove:
Size: Compressed: 117K, uncompressed: 290K.
Installing package jove...
Executing installation script for package jove...
Done installing package jove.
Auto-installing package ==>manpages<== Priority: [recommended]
manpages: Man-pages 1.24
manpages:
manpages: Man pages are online documentation for Linux. This package includes
manpages: many section 1, 2, 3, 4, 5, 7, and 8 man pages for Linux.
manpages:
manpages: The man-pages distribution is maintained by Andries E. Brouwer with
manpages: the help of more writers, editors, and other contributors than we
manpages: can name here. (See the list in /usr/man/man-pages-1.24.Announce)
manpages:
Size: Compressed: 941K, uncompressed: 1670K.
Installing package manpages...
Done installing package manpages.
Auto-installing package ==>mc<== Priority: [optional]
mc: mc-4.5.39
mc:
mc: The Midnight Commander is a Norton Commander clone, a program that
mc: manipulates and manages files and directories. Useful, fast, and has
mc: color displays on the Linux console. Mouse support is provided
mc: through the gpm mouse server. This is a lightweight compilation of
mc: mc for use on the text console or in an xterm.
mc:
mc: mc was written by Miguel de Icaza and Mauricio Plaza.
mc:
Size: Compressed: 1238K, uncompressed: 3250K.
Installing package mc...
Executing installation script for package mc...
Done installing package mc.
Auto-installing package ==>mt_st<== Priority: [optional]
mt_st: mt-st-0.4 - controls magnetic tape drive operation
mt_st:
mt_st: This mt originates from BSD NET-2. Some Linux SCSI tape-specific
mt_st: ioctls have been added to the original source and the man page has
mt_st: been updated. Although this mt program is tailored for SCSI tape, it
mt_st: can also be used with the QIC-02 driver and hopefully with other Linux
mt_st: tape drivers using the same ioctls (some of the commands may not work
mt_st: with all drivers).
mt_st:
Size: Compressed: 10K, uncompressed: 30K.
Installing package mt_st...
Executing installation script for package mt_st...
Done installing package mt_st.
Auto-installing package ==>quota<== Priority: [optional]
quota: Linux disk quota utilities (1.70)
quota:
quota: An implementation of the diskquota system for the Linux operating
quota: system -- keeps those greedy users from gobbling up 100% of the hard
quota: drive space. Should work for all filesystems because of integration
quota: into the VFS layer of the operating system. This system is a heavily
quota: modified version of the Melbourne quota system by Robert Elz, which
quota: uses both user and group quota files. Edvard Tuinder <ed@ow.org> and
quota: Marco van Wieringen <mvw@planets.ow.org> are responsible for the
quota: additional code and porting effort to make quotas work on Linux.
quota:
Size: Compressed: 35K, uncompressed: 100K.
Installing package quota...
Executing installation script for package quota...
Done installing package quota.
Auto-installing package ==>raidtool<== Priority: [optional]
raidtool: raidtools-0.41
raidtool:
raidtool: Utilities required to support RAID-1/4/5. Its main goal is to group
raidtool: several disks or partitions together, making them look like a single
raidtool: block device. For updated information on the RAID-1/4/5
raidtool: personalities, please read the QuickStart.RAID in
raidtool: /usr/doc/raidtools-0.41.
raidtool:
Size: Compressed: 33K, uncompressed: 100K.
Installing package raidtool...
Executing installation script for package raidtool...
Done installing package raidtool.
Auto-installing package ==>rpm<== Priority: [optional]
rpm: rpm-3.0.2
rpm:
rpm: RPM is a tool from Red Hat Software used to install and remove
rpm: packages in the .rpm format. When installing RPM packages on
rpm: Slackware, you may need to use the --nodeps and --force options.
rpm: Before installing any binary package, it's wise to examine it to
rpm: see what it's going to do, and if it will overwrite any files. You
rpm: can use rpm2tgz to convert .rpm packages to .tgz packages so you
rpm: can look them over. (in fact, it might be safer to install the
rpm: resulting .tgz with 'installpkg')
rpm:
Size: Compressed: 1006K, uncompressed: 2640K.
Installing package rpm...
Executing installation script for package rpm...
Done installing package rpm.
Auto-installing package ==>sc<== Priority: [optional]
sc: The 'sc' spreadsheet. (v. 6.21)
sc:
sc: This is a simple spreadsheet program.
sc:
Size: Compressed: 101K, uncompressed: 240K.
Installing package sc...
Done installing package sc.
Auto-installing package ==>seejpeg<== Priority: [recommended]
seejpeg: seejpeg-1.6.1
seejpeg:
seejpeg: Seejpeg is an image viewer which uses SVGAlib. In spite of the name,
seejpeg: it understands the JPEG, GIF, PPM, BMP and TARGA formats. One nice
seejpeg: feature of seejpeg is that it automatically decides on the best video
seejpeg: mode to use based on the image being displayed and the video card
seejpeg: being used.
seejpeg:
seejpeg: Evan Harris is the author of seejpeg.
seejpeg:
Size: Compressed: 52K, uncompressed: 120K.
Installing package seejpeg...
Done installing package seejpeg.
Auto-installing package ==>sox<== Priority: [recommended]
sox: sox-12.15
sox:
sox: Sox is a universal sound converter, player, and recorder. It can
sox: play, record, and convert between several sound file formats such as
sox: .au, .snd, .voc and .wav.
sox:
Size: Compressed: 82K, uncompressed: 190K.
Installing package sox...
Executing installation script for package sox...
Done installing package sox.
Auto-installing package ==>sudo<== Priority: [optional]
sudo: sudo-1.5.9p4-1
sudo:
sudo: 'sudo' is a command that allows users to execute some commands as
sudo: root. The /etc/sudoers file (edited with 'visudo') specifies which
sudo: users have access to sudo and which commands they can run. 'sudo'
sudo: logs all its activities to /var/log/ so the system administrator
sudo: can keep an eye on things.
sudo:
Size: Compressed: 87K, uncompressed: 210K.
Installing package sudo...
Executing installation script for package sudo...
Done installing package sudo.
Auto-installing package ==>texinfo<== Priority: [recommended]
texinfo: GNU texinfo-3.12
texinfo:
texinfo: 'Texinfo' is a documentation system that uses a single source file to
texinfo: produce both on-line information and printed output. Using Texinfo,
texinfo: you can create a printed document with the normal features of a book,
texinfo: including chapters, sections, cross references, and indices. From the
texinfo: same Texinfo source file, you can create a menu-driven, on-line Info
texinfo: file with nodes, menus, cross references, and indices.
texinfo:
texinfo: This package is needed to read the documentation files in /usr/info.
texinfo:
Size: Compressed: 405K, uncompressed: 770K.
Installing package texinfo...
Done installing package texinfo.
Auto-installing package ==>vim<== Priority: [optional]
vim: Version 5.5 of Vim: Vi IMproved
vim:
vim: Vim is an almost compatible version of the UNIX editor vi. Only the
vim: 'Q' command is missing (you don't need it). Many new features have
vim: been added: multi level undo, command line history, filename
vim: completion, block operations, etc.
vim:
Size: Compressed: 1263K, uncompressed: 3830K.
Installing package vim...
Executing installation script for package vim...
Done installing package vim.
Auto-installing package ==>workbone<== Priority: [optional]
workbone: Workbone 2.31-5
workbone:
workbone: Text-based audio CD player.
workbone:
Size: Compressed: 10K, uncompressed: 30K.
Installing package workbone...
Done installing package workbone.
Auto-installing package ==>zsh<== Priority: [optional]
zsh: zsh version 3.0.6
zsh:
zsh: Zsh is a UNIX command interpreter (shell) which of the standard shells
zsh: most resembles the Korn shell (ksh), although it is not completely
zsh: compatible. It includes enhancements of many types, notably in the
zsh: command-line editor, options for customizing its behavior, filename
zsh: globbing, features to make C-shell (csh) users feel more at home and
zsh: extra features drawn from tcsh (another 'custom' shell). Zsh was
zsh: written by Paul Falstad.
zsh:
Size: Compressed: 358K, uncompressed: 610K.
Installing package zsh...
Executing installation script for package zsh...
Done installing package zsh.
INSTALLING A LINUX KERNEL
Now you should verify that a Linux kernel is installed to your new Linux
system. If you've already installed a generic Linux kernel from the A series,
then you can skip this step. Be advised, however, that your best option is
probably to install the kernel from your installation bootdisk. If it got
you this far without problems, you know it will work. If additional driver
support is required after installation, it can be added in the form of loadable
kernel modules by editing your /etc/rc.d/rc.modules file.
1 - Install the kernel from the installation bootdisk.
2 - Install a ZIMAGE, BZIMAGE, or VMLINUZ kernel image from an MS-DOS
formatted floppy disk.
3 - Install a kernel from the Slackware CD-ROM.
4 - Skip this step -- do not install a Linux kernel. (NOTE: A Linux kernel
must have already been installed, or you'll have to use your installation
bootdisk or LOADLIN to start your system)
Which option would you like (1/2/3/4)? 4
It is recommended that you make a boot disk.
Would you like to do this ([y]es, [n]o)? n
MODEM/MOUSE CONFIGURATION
Modem Setup:
A link in /dev will be created from your callout device (ttyS0, ttyS1,
ttyS2, ttyS3) to /dev/modem. You can change this link later if you put
your modem on a different port.
Would you like to set up your modem ([y]es, [n]o)? n
Mouse Setup:
A link will be created in /dev from your mouse device to /dev/mouse.
You can change this link later if you switch to a different type of
mouse.
Would you like to set up your mouse ([y]es, [n]o)? n
Do you have a CD-ROM ([y]es, [n]o)? n
Would you like to try out some custom screen fonts ([y]es, [n]o)? n
LILO INSTALLATION
LILO (the Linux Loader) is the program that allows booting Linux directly from
the hard drive. To install, you make a new LILO configuration file by creating
a new header and then adding at least one bootable partition to the file. Once
you've done this, you can select the install option. Alternately, if you
already have an /etc/lilo.conf, you may reinstall using that. If you make a
mistake, just select (1) to start over.
1 -- Start LILO configuration with a new LILO header
2 -- Add a Linux partition to the LILO config file
3 -- Add an OS/2 partition to the LILO config file
4 -- Add a DOS partition to the LILO config file
5 -- Install LILO
6 -- Reinstall LILO using the existing lilo.conf
7 -- Skip LILO installation and exit this menu
8 -- View your current /etc/lilo.conf
9 -- Read the Linux Loader HELP file
Which option would you like (1 - 9)? 7
Configuration of 'gpm':
"gpm" is a program that allows you to do cut and paste on the
virtual consoles using a mouse. If you choose to have it run at boot
time, the line "gpm -m /dev/mouse -t none " will be added to
your /etc/rc.d/rc.gpm.
Running gpm with a bus mouse can cause problems with XFree86.
If XFree86 refuses to start and complains that it cannot open the
mouse then you might want to comment the line out of /etc/rc.d/rc.gpm.
Would you like to add "gpm -m /dev/mouse -t none " to /etc/rc.d/rc.gpm
so that gpm will load at boot time ([y]es, [n]o)? n
Would you like to configure your timezone ([y]es, [n]o)? n
WARNING: NO ROOT PASSWORD DETECTED
There is currently no password set on the system administrator account
(root). It is recommended that you set one now so that it is active the
first time the machine is rebooted. This is especially important if
you're using a network enabled kernel and the machine is on an Internet
connected LAN.
Would you like to set a root password (y[es]/n[o])? y
Changing password for root
Enter the new password (minimum of 5, maximum of 127 characters)
Please use a combination of upper and lower case letters and numbers.
New password:
Bad password: too short.
Warning: weak password (enter it again to use it anyway).
New password:
Re-enter new password:
Password changed.
Press [enter] to continue:
Linux installation is complete.
You may now reboot your computer by pressing control+alt+delete.
If you installed LILO, remove the boot disk from your computer
before rebooting.
/var/adm/mount: Read-only file system
# cd /mnt
# tar xpf /initfiles.tar
# halt
Broadcast message from root (tty1) Thu Aug 31 16:56:08 2000...
The system is going down for system halt NOW !!
#
Back to the main console for the shutdown:
passwd[4451]: password for `root' changed by `root'
INIT: Switching to runlevel: 0
INIT: Sending processes the TERM signal
INIT: Sending processes the KILL signal
INIT: no more processes left in this runlevel
This is a list of the UML-specific command line arguments, plus a few
generic ones which deserve mention here.
--help
This causes UML to print a usage message and exit.
--version
This causes UML to print its version and exit.
--showconfig
This causes UML to print the config file it was built with and exit.
con
con=channel attaches one or more UML consoles to the
named channel. The format of the channel is described
here.
debug
Starts up the kernel under the control of gdb. See the
kernel debugging
tutorial and the
debugging session pages for more information. Another form of
this switch is debug=go which is the same as debug
except that the kernel runs instead of stopping at the beginning of
start_kernel.
If you're using ddd to debug UML, you will want to specify
debug=parent as well as gdb-pid (see below).
This switch is specific to tt
mode and has no effect in skas mode.
debugtrace
Causes the tracing thread to pause until it is attached by a debugger
and continued. This is mostly for debugging crashes early during
boot, and should be pretty much obsoleted by the debug switch.
This switch is specific to tt
mode and has no effect in skas mode.
dsp
dsp=host dsp tells the UML sound driver what the
filename of the host dsp is so that it can relay to it. The default
is "/dev/sound/dsp".
eth
ethn=host interface enables a virtual ethernet
device inside UML. See the
networking HOWTO
for more information on setting up UML networking.
fakehd
Causes the ubd device to put its partition information in
/proc/partitions under the device name "hd" rather than "ubd". Again,
this is to fake out installation procedures which are overly picky in
their sanity-checking.
fake_ide
fake_ide causes the ubd driver to install realistic-looking
entries into /proc/ide. This is useful for convincing some
distribution installation procedures to run inside UML.
gdb-pid
gdb-pid=pid, when used with debug, specifies the pid of
an already running debugger that UML should attach to. This can be used
to debug UML with a gdb wrapper such as emacs or ddd, as well as with debuggers
other than gdb. See the
debugging page for more information.
This switch is specific to tt
mode and has no effect in skas mode.
honeypot
honeypot causes UML to rearrange its address space in order to
put process stacks in the same location as on the host. This allows
stack smash exploits to work against UML just as they do against the
host. This option enables jail, since it is most unlikely that
a honeypot UML should run without it enabled.
This switch is specific to tt
mode and has no effect in skas mode. Honeypots should be run
in skas mode anyway, since they will perform far better, and the
security model is much simpler, making it less likely that there will
be exploitable bugs that will allow an attacker to break out.
initrd
initrd=image sets the filename of the initrd image that
UML will boot from.
iomem
iomem=name,file makes file available to be
mapped by a driver inside UML. See
this page for more information.
jail
jail enables protection of UML kernel memory from UML
processes. This is disabled by default for performance reasons.
Without it, it is fairly simple to break out of UML by changing the
right pieces of UML kernel data.
This switch is specific to tt
mode and has no effect in skas mode. skas mode doesn't have
the same problems with processes being able to break out to the host,
so this switch isn't needed. Effectively, 'jail' mode is always
enabled in skas mode.
mconsole
mconsole=notify:socket asks the mconsole driver to send
the name of its socket to the Unix socket named by this switch. This
is intended for the use of scripts which want to know when they can
start using the mconsole and what socket they should send commands to.
mem
mem=size controls how much "physical" memory the kernel
allocates for the system. The size is specified as a number followed by one
of 'k', 'K", 'm', 'M", which have the obvious meanings. This is not related
to the amount of memory in the physical machine. It can be more, and
the excess, if it's ever used, will just be swapped out.
In its default configuration, UML has a maximum physical memory size
of just under 512M. If you specify more than that, it will be shrunk,
and a warning printed out. If your UML is configured with highmem
support (CONFIG_HIGHMEM) enabled, then any physical memory beyond what
can be directly mapped in to the kernel address space will become
highmem. In this case, the current limit on UML physical memory is 4G.
Something to note if you have a small /tmp is that UML creates a file
in /tmp which is the same size as the memory you specified. It is not
visible because UML unlinks it after creating it. This can cause /tmp
to mysteriously become full. UML respects the TMP, TEMP, and TMPDIR
environment variables, so you can avoid this problem by specifying an
alternate temp directory.
Something else to note is that UML is noticably faster with a tmpfs
/tmp than with a disk-based /tmp such as ext2 or ext3.
mixer
mixer=host mixer tells the UML sound driver what the
filename of the host mixer is so that it can relay to it. The default
is "/dev/sound/mixer".
mode=tt
mode=tt forces UML to run in tt mode (see
this page for the
details) even when skas support is built in to UML and the host.
Using this switch without both tt and skas modes built in to UML will
have no effect aside from producing a warning during boot.
ncpus
ncpus=number tells an SMP kernel how many virtual processors to
start. This switch will only take effect if CONFIG_UML_SMP is enabled
in the UML configuration.
ssl
ssl=channel attaches one or more UML serial lines to the
named channel. The format of the channel is described
here.
root
root=root device is actually used by the generic kernel in
exactly the same way as in any other kernel. If you configure a number of
block devices and want to boot off something other than ubd0, you would use something like:
root=/dev/ubd5
tty_log_dir
tty_log_dir=directory changes the directory to which UML
writes tty logging files. This requires that tty logging be
configured into UML. See the
tty logging page
for more details.
tty_log_fd
tty_log_fd=file descriptor causes tty logging records to
be written to the file descriptor specified. This descriptor must be
opened before UML is run and passed in to UML. See the
tty logging page
for more details.
ubd
ubd=number causes the ubd device to take over a different major
number than the one assigned to it. This is useful for making it appear to
be an "hd" device.
ubd
ubdn=filename is used to associate a device with a file
in the underlying filesystem. Usually, there is a filesystem in the file,
but that's not required. Swap devices containing swap files can be specified
like this. Also, a file which doesn't contain a filesystem can have
its contents read in the virtual machine by running dd on the device.
n must be in the range 0 to 7. Appending an 'r' to the number will
cause that device to be mounted read-only. Appending an 's' will
cause that device to do all IO to the host synchronously. If both 'r'
and 's' are specified, it must be as 'rs'.
Inside UML, if you are not using devfs, the devices are accessible
with minor numbers 0, 16, ..., with the other minor numbers being used
for partitions. So, the device that is ubd1 on the UML command line
becomes /dev/ubd16 inside UML.
ubd
ubdn=cow-file,backing-file is used to layer a COW
file on another, possibly readonly, file. This is useful in a number
of ways. See this
page for all the details.
umid
umid=name is used to assign a name to a virtual machine. This
is intended to make it easy for UIs to manage multiple UMLs. Currently, the
only effect of this is that UML writes its tracing thread pid in
/tmp/uml/name.
uml_dir
uml_dir=directory sets the directory in which UML will
put the umid directory, which in turn will contain the pid file and
mconsole socket.
umn
umn=ip-address sets the ip address of the host side of the slip
device that the umn device configures. This is necessary if you want to set up
networking, but your local net isn't 192.168.0.x, or you want to run
multiple virtual machines on a network, in which case, you need to
assign different ip addresses to the different machines. See the
networking tutorial for more information.
xterm
xterm=terminal emulator,title switch,exec
switch allows you to specify an alternate terminal emulator for
UML to use for the debugger, consoles, and serial lines. terminal
emulator is the emulator itself, title switch is the switch
it uses to set its title, and exec switch is the switch it uses
to specify a command line to exec. The two switches need to have the
same syntax and semantics of xterm's "-T" and "-e". The default
value is "xterm=xterm,-T,-e". To use gnome-terminal, you would
specify "xterm=gnome-terminal,-t,-x". If any fields are left blank,
the default values will be used. So, to use "myxterm", which has the
same switches as xterm, "xterm=myxterm" will suffice.
Hosted at
user-mode-linux-doc-20060501/tests.html 0000644 0000000 0000000 00000046715 12742461304 014507 0 ustar
The UML test suite
The UML test suite is a set of Perl scripts which I use as a
regression test on new UML patches. It comprises a test harness,
which organizes and runs the suite, and the suite itself, which is a
directory hierarchy which groups similar tests.
The tests to be done during a particular test run are specified by the
parent directory, if you want all of the tests in that directory to
run, an individual test, or any combination of directories and files.
The tests are able to specify the properties of the UML in which they
are to run. The test harness compiles and runs UMLs according to the
test requirements, trying to minimize the number of builds and boots
required to satisfy all of the tests' requirements.
The tests vary widely in what they test and how they work. The
simplest ones run a command (which they may have compiled themselves)
and check the return value. Others run gdb and make it step through a
test program to check that breakpoints work correctly. These use an
already-booted UML, and don't need any special setup of the UML.
In contrast, there are tests which have requirements of the UML. For
example, the boot_filesystems test checks that a user-specified set of
filesystems boots correctly. So, it requests an unbooted UML for its
use, but puts no more requirements on it. Others test various
configurable options, and request a UML with certain options enabled,
which they boot.
Finally, there are tests which stress the kernel generally in order to
make sure that it is stable under a hard workload. The kbuild test,
which runs a kernel build, is an example of this.
Why you should run it
The main reason would be to do a sanity-check on UML when compiled
locally, with whatever unusual config options you favor, and run on
your hardware. UML is somewhat sensitive to toolchain changes, and,
to a less extent, to the host hardware. Users regularly shake out
bugs that don't appear during development, and these are often exposed
by different toolchains and hardware.
Secondly, if UML fails mysteriously somehow, a test may exercise the
same bug. In this case, the test is likely to be much more specific
than your workload, and the search for the bug much more constrained.
It would also be a help to the UML project in general to have the
tests run as often as possible, and on as a great a variety of hosts
as possible. This will expose bugs, and get them fixed, more quickly.
How to run it
First, download the latest version from the test suite section of
the UML download page.
Set up a ~/.umltest, which is the config file needed to tell the
harness about the external resources, such as filesystems, and other
information it will need in order to run. The file contains a perl
hash, with different types of information under different keys. If
you don't know perl, don't panic, because the syntax is simple
enough. Mine looks like this:
The kernel_pool tells the harness the location of a UML source pool.
This will be where it builds whatever UMLs it needs.
The uml section specifies the default UML command. This will be used
for any tests which don't have special requirements. The ubd hash
within that is a mapping of device numbers to files, and this one will
turn into "ubd0=/home/jdike/roots/debian_22" on the command line. The
prompts hash specifies a regular expression which matches the shell
prompts for the filesystems that will be booted.
The tests section provides test-specific information. The keys here
are the filenames of the tests under the tests/ directory, and without
the ".pl" extension. They can ask for arbitrary information here. I
specify one filesystem for the boot_filesystems test to boot, and an
image of a filesystem containing a kernel source tree for the kernel
build test.
Now, the tests can be run as follows
host% perl test.pl
This will run the full test suite, reporting an overall status when
it's done. A full log of the run will appear in tests.log. This will
contain all output from the UMLs, including boot logs and anything the
tests do.
Specific tests can be run by adding them to the command line
perl test.pl stress/kbuild
Adding a directory to the command line will cause all of the tests
within the that directory to be run. So,
perl test.pl functional
will run all the tests under functional/, but nothing else.
How to add tests
The tests are all under the "tests" directory, each in a separate perl
file. Here is a very simple test, commented:
# Virtually all tests will need these two modules. The first defines
# the Test object, an instance of which is returned from here. The
# second is a library of useful utilities which essentially all of the
# tests use.
use UML::Test;
use UML::Testlib;
# strict is good
use strict;
# This function is the actual test. It will be run inside a booted
# UML, at the shell. run_exit_0 is from TestLib. It runs the dd
# command within the UML object given to the test, returning if the
# command had an exit status of 0, and dying if not. In this case,
# the harness will trap that and report the failure.
sub run {
my $test = shift;
my $uml = shift;
run_exit_0($uml, "dd if=/dev/mem of=/dev/null");
}
# This constructs the Test object and returns it. This specifies that
# this test be run in a booted UML (state => "up") and that the test
# is contained in the 'run' function.
UML::Test->new(state => "up", run => \&run);
This illustrates the two essential features of a test
The function which executes the actual test
The construction of the test object, which specifies the test function
and any requirements that the test makes on the UML.
The test can also request an unbooted UML with 'state => "up"', if the
purpose of the test is to check that UML boots. It can also request a
UML with special configuration options. The initrd test does this
like so:
The kconfig option requests that it be given a UML with CONFIG_MMAPPER
set to "y", and the config option adds 'iomem=mmapper,$file' to the
UML command line.
A number of people have helped this project in various ways, and this page
gives recognition where recognition is due.
If you're listed here and you would prefer a real link on your name,
or no link at all, instead of the despammed email address pseudo-link,
let me know.
If you're not listed here and you think maybe you should be, please let
me know that as well. I try to get everyone, but sometimes my bookkeeping
lapses and I forget about contributions.
Lennert Buytenhek - Contributed various patches, a rewrite of the
network driver, the first implementation of the mconsole driver, and
did the bulk of the work needed to get SMP working again.
Yon Uriarte - Fixed the TUN/TAP network backend while I slept.
Adam Heath - Made a bunch of nice cleanups to the initialization code,
plus various other small patches.
Matt Zimmerman - Matt volunteered to be the UML Debian maintainer
and is doing a real nice job of it. He also noticed and fixed a
number of actually and potentially exploitable security holes in
uml_net. Plus the occasional patch. I like patches.
James McMechan - James seems to have taken over maintenance of the ubd
driver and is doing a nice job of it.
Chandan Kudige - wrote the umlgdb script which automates the reloading
of module symbols.
Steve Schmidtke - wrote the UML slirp transport and hostaudio drivers,
enabling UML processes to access audio devices on the host. He also
submitted patches for the slip transport and lots of other things.
SGI (and more specifically
Ralf Baechle) gave me an account on
oss.sgi.com. The bandwidth
there made it possible to produce most of the filesystems available on
the project
download page .
Laurent Bonnaud took the old grotty
Debian filesystem that I've been distributing and updated it to 2.2.
It is now available by itself here .
Rik van Riel gave me some ftp space on ftp.nl.linux.org so I can
make releases even when Sourceforge is broken.
Rodrigo de Castro looked at my broken pte code and told me what
was wrong with it, letting me fix a long-standing (several weeks) and
serious set of bugs.
Chris Reahard built a specialized root filesystem for running a
DNS server jailed inside UML. It's available from the
download page in the
Jail Filesysems section.
Hosted at
user-mode-linux-doc-20060501/todo.html 0000644 0000000 0000000 00000070635 12742461304 014310 0 ustar
The Current TODO list
Since about Feb 2001, I've been maintaining a TODO list and posting it to the
devel list occasionally.
Not done
Packaging
Functionality
Bugs
My attempts to run the Debian install procedure hung UML while it was
probing disks. I never figured out what was happening.
Make UML build without warnings
Make sure that each clock tick gets counted
Figure out the hostfs crash that Larent Bonnaud is seeing
make 'gdb=pty' work
protect kernel memory from userspace
Figure out why the io_thread loses its parent and its connection to the kernel
Disable SIGIO on file descriptors which are already being handled.
This will cut off some interrupt recursion.
Figure out why gdb can't use fd chan (blinky@gmx.net)
Figure out why repeated 'tail /dev/zero' with swap causes process segfaults
Set SA_RESTART on SIGPROF
Replace dentry_name with d_name
Dynamically allocate all driver descriptors
Make slip_tramp conditional in slip_close
many du / causes slab cache corruption
Adding an eth device via the mconsole (and probably the command line)
won't necessarily configure the device that was named
Figure out why init/version.c, init/main.c, and arch/um/main.c don't
get coverage data.
Fix the ubd rounding bug spotted by Steve Schmidtke and Roman Zippel.
gdb should be the last thing to be shut down so that breakpoints can
be set late in the shutdown process.
Have the __uml_exitcall handlers check to see if they need to do anything.
Find the race that causes the kernel to run before the debugger is running,
and which then causes a panic and segfault.
Tests that need writing
Build and load some modules to check for unexported symbols
Swap testing - qsbench, low-memory kernel build
Rerun some existing tests through hostfs
Figure out why gdb inside UML sometimes misses breakpoints.
^C doesn't work well in a busy loop in __initcall code. I've seen the
process die with a SIGINT as well as have the SIGINT arrive to a
sleeping process, panicing the tracing thread.
When setting a conditional breakpoint on __free_pages in
free_all_bootmem_core and continuing, UML hangs.
Figure out what to do with __uml_setup in modules. A lot of these should
end up being normal __setup, since they don't have to happen before the
kernel runs.
Figure out why UML died nastily after ^C-ing it and hitting it in userspace.
Telnetting to a console when in.telnetd is non-executable produces an
error and a crash from a sleeping process segfaulting.
Single-stepping when a signal has arrived doesn't work. gdb
singlesteps, sees the signal, and single-steps with the signal. The
signal is handled immediately, stepping the process into the handler.
The original instruction was never executed, so when gdb puts the
breakpoint back and the handler returns, the breakpoint is hit again.
The _to_user routines refer to user data without using copy_to_user.
How to cause a strange userspace segfault - run the new UML inside UML
under gdb. Put a breakpoint on create_elf_tables. 'finish', 'next',
segfault.
With an old port-helper hanging on to a port, running a UML which want those
ports causes all of the consoles and serial lines to output their login
prompts to stdout.
Assigning console and serial line devices to files doesn't work cleanly.
Make sure that irqs are deactivated or freed when their descriptors are
closed.
Things to audit:
Make sure that destructors exactly undo everything the constructor does
Set FD_CLOEXEC on any descriptors that don't need to be passed across execs.
Make sure any protocols are 64-bit clean - this means not using ints, longs,
etc. Also, maybe enums are bad.
port_kern.c has port_remove_dev and port_kern_free which do almost the
same things.
Make sure that return values are actually used.
skas4 things - remove the extra two context switches per system call,
make sure it compiles with CONFIG_PROC_MM off, implement the mm
indirector, move PTRACE_LDT to /dev/mm, fix PTRACE_FAULTINFO to return
trap type and err, allow the environment, argument, etc pointers of an
mm to be set.
Confirm fix
Get either Myrtal Meep or Matt Clay
to confirm (or not) that they can no longer crash UML with
Apache/Perl/MySQL. This will probably be fixed with the new network drivers.
Done
Packaging
Put together a UML deb builder
Figure out how to ship the UML test suite
Figure out how to make a module tar
file with the right structure
It currently sucks. What I want is an RPM and a deb for each release
containing:
the kernel
HOWTO
user-mode network tools
test harness
test suite
maybe a small root filesystem like the tomsrtbt
modules
mkrootfs
Anything else? I also need suggestions for locations for all these things.
Functionality
Add a mechanism for rereading module symbols when the module is reloaded.
Integrate Greg Lonnon's I/O space emulation
Make the socket channel work
Figure out what to do
about the network driver. There are duelling drivers from Jim Leu and
Lennert Buytenhek, and I need to figure out how to merge the best bits from
them into a single driver, or split them into two very distinct drivers.
Extract Patrick Schaaf's cool new block driver from him.
Remove the tty mode save/restore from main and put it in the file descriptor
channel code. (Boria Feigin)
Added more stuff to
/proc/cpuinfo (mistral)
Allow gdb to be attached to a
host device in the same way that consoles and serial lines can
modify_ldt implemented
(patch from Lennert Buytenhek)
Add support for root hostfs
Added --help (patch from
Yves Rougy)
Bugs
I occasionally see one of the loggers
dying from a SIGTERM during boot because of a kill that's somehow directed
at the wrong process.
I need to figure out why some people
see 'Unexpectedly got signal 4 in signals'. This has been reported a
couple of times, but I've never got any information that would help me
debug it.
Implement copy_{from,to}_user using
setjmp/longjmp.
Fix gcov and gprof support
Figure out why the umn device won't serve more than one packet to remote machines.
Figure out how to make
-D_FILE_OFFSET_BITS=64 work with modules (fix from Lennert which explicitly
uses the *64 interfaces)
Figure out why ^S and ^Q don't work on the console (patch from Livio)
Stop the hangs caused by someone disconnecting from a virtual console
umn needs to destroy the slip device
when it's no longer needed (JS)
clean up the umn_set_addr logic
Make sure that readonly hostfs really
is readonly (James McMechan)
Figure out the wait_for_page hangs
I used to see syslog occasionally hang,
but haven't in a long time. It happened again, and it turns out I was
enabling signals when they shouldn't be, allowing the timer to interrupt a
sensitive part of do_IRQ.
gdb doesn't find the kernel if it isn't called 'linux' (Brian J. Murrell)
Have the ubd driver figure out the
block device size with BLKGETSIZE if necessary (Lennert Buytenhek, marc)
Make sure it boots in the presence of IO
redirection and piping (JS)
Rebooting a UML under debug won't kill the old gdb
Remove assumption that argv is
contiguous (Boria Feigin)
Have hostfs copy user data with
copy_{to,from}_user.
Put in Lennert's utsname patch
(Lennert Buytenhek)
Put in Lennert's ubd LFS patch
(Lennert Buytenhek)
The kernel shouldn't hang when debug is
requested and it wasn't compiled with CONFIG_PT_PROXY.
Make empty_zero_page and empty_bad_page
reserved (Rik)
Increase COMMAND_LINE_SIZE to something
bigger (JS)
Put breakpoints on panic and stop
Make hostfs figure out external filenames
by walking up the dentry->parent tree rather than having inodes store them.
Figure out why hostfs misses files in large
directories
Make sure init can exec (Al Viro)
Fix the problems that mistral is
seeing. This is three bugs - the sleep hang, a bug in
sigio_handler (both fixed), plus the sigreturn bug.
Reimplement process signal delivery to
eliminate the race when the process handler returns
Various fixes and cleanups in hostfs (Al Viro)
Remove real_mm from thread and replace
it with some unused pte bits
Fix /proc/cmdline
(prose from Henrik Nordstrom, code from Greg Lonnon)
drivers Makefile cleanup
(patch from Greg Lonnon)
tty_flip_buffer_push fixlet patch
from (Gordon McNutt)
The ubd device should reset openflags between mounts (jfreak)
Figure out the sleep hang. It
happens rarely. Somehow, irq_desc[0].flags gets stuck at 5 which tells do_IRQ
that the interrupt is already being handled.
Stop ^C in gdb from segfaulting processes if it hits in userspace.
Figure out why swapping causes
process segfaults and nasty mm messages.
Fix the build so that modules_install
drops modules in the proper hierarchy (fix from Henrik Nordstrom)
Got rid of hostfs_llseek (patch from
stewart and Henrik Nordstrom)
gdb now ignores SIGWINCH. (patch
from James Stevenson)
Fixed ubd stats (patch from James Stevenson)
UML should run on kernels with
a 2G/2G address space split (blinky)
Fix the segv bug where
a --x page is never fixed because thread.starting_exec is set (mistral) -
mistral hasn't complained about this in ages
Fix 'cat /proc/kmsg' (Torsten Fink)
Fix crash if someone types at the console too soon (patch from mistral)
Change __SMP__ to CONFIG_SMP (Niels Kristian Bech Jensen)
Include config.h where necessary (Niels Kristian Bech Jensen)
Fix a couple allocation buglets in hostfs (Henrik Nordstrom)
Implement internal system calls correctly (Roman Zippel)
Allow the time to be changed (Frank Klingenhoefer) (patch from Livio Soares)
The ptrace proxy should handle wait correctly for non-UML subprocesses of gdb
Figure out why UML flunks the f00f test
Make the build work when CONFIG_MCONSOLE is
off
Provide some kind of reasonable error
message when the root filesystem isn't writable.
Figure out why hostfs panics when it's on an nfs directory on the host
(David Coulson)
Remove the chdir from the umid setup
Figure out why the .deb build fails
checksumming - it was because hostfs reads went through the page cache
and writes bypassed it
Figure out why diffing identical kernel
pools produces diffs - can't reproduce this, so I have to assume that it's
fixed somehow
Make sure that gdbs get killed properly
Stop dev_ip_addr from crashing if the
interface has no IP addresses
Figure out why the Debian ping is slower than the Slackware ping
set_umid should only set the umid
COW headers should have absolute pathnames
in them or backing file path names should be possibly relative to the COW
file's directory
Figure out why some early breakpoints don't
work. I.e. Setting a bp in do_initcalls or bdflush and continuing from
start_kernel misses the bp, but continuing from rest_init hits it.
Print out a decent error message when mcast setup fails because there
are no multicast devices on the host
Suppress the error message from uml_net when route fails because
there's no eth0 on the host
See if it's possible to have uml_net know when an eth device changes
its IP address.
Figure out why apt-get install ntp crashes UML.
Version the mconsole and uml_router interfaces
Fix the error messages when a backing file doesn't exist, and also when the
COW file exists, but is empty.
Put -fno-common back
Fix gdb
Fix daemonized operation
Implement thread_saved_pc
Run sysrq stuff in interrupt context
Make it possible to ^Z and bg UML
Make core dumping work
many du / causes process segfaults
ddd doesn't work with gdb-pid
Removing a block device with the mconsole doesn't remove its entry in
/proc/partitions
Removing ethertap or TUN/TAP eth devices produces odd error messages in
UML
Figure out why running UML under an identical UML hangs the outer one
Figure out why running a properly built UML under UML doesn't work
gdbs aren't getting killed properly again
Setting up a tuntap interface with no gate address will hang UML when
the interface is brought up
Figure why processes segfault under extreme load, like a 'make -j'
kernel build
Figure out why leaning on the space bar when top is running causes an FPE.
Figure out how to let arches ifdef out PTRACE_[SG]ETREGS
Put UML temporary files (except maybe the mmap files) in the user's
home directory.
Fix the bogomips calculation.
Figure out how to implement thread private pages.
Figure out why ^S and ^Q don't work on the console (phillips)
When a non-root ubd device is COWed, the bitmap isn't mapped in for some
reason.
The mconsole driver should support C-A-D if it doesn't already.
'i reg' to gdb inside UML doesn't print the fp registers
Improve the formatting when a error message is returned from uml_net,
i.e. when TUN/TAP is requested, but uml_net doesn't support TUN/TAP.
Get the fp registers in core files correctly.
gdb should be able to see ^C immediately
Byte-swap the ubd bitmap (Roman Zippel)
The register state in the sigcontext passed in to a signal handler should
be copied into the process registers when the handler returns.
When UML halts, any external debugger needs to be told that it exited so
it doesn't get confused about the child suddenly disappearing.
setup_stack should use copy_to_user.
Using strace as an external debugger doesn't work. strace sees no system
calls starting at the delay calibration.
When a new COW file is created and the backing file doesn't exist, the
error message implies the COW file doesn't exist, plus the COW file is
created with size 0.
mconsole clients need to pass absolute pathnames into UML.
Make sure there are no references to errno in kernel code
Complain when a channel type is requested but not configured in
Get the tracing thread out of the business of running trampolines
disable_chan needs to free up all SIGWINCH info so that logging out of
a port console doesn't produce nasty-looking errors.
Also the woody filesystem produces nasty errors when logging in through
telnet.
close_chan should call free_irq, not free_irq_by_fd
Running the ists debian fs with devfs=nomount produces a 'Freeing free IRQ'.
Look at the initrd dependencies. UML doesn't compile when initrd support
is disabled.
Multiple opens of a pty console cause irq registrations to pile up.
Sanity-check arguments in the network *setup routines. Ethertap in
particular needs checking. dev->user should remain NULL if there's
an error. Also sanity-checking in uml_net is needed.
The uml_net entry points should check their argcs.
Figure out why large pastes into consoles get truncated.
'next' across a statement which segfaults causes wait_for_stop to get
confused by the segfault.
Make sure "none" is commutative with the other channel initializations.
It's still possible for disconnecting from a UML console to put that
device into an infinite poll loop after it's closed.
Giving the wrong pid to gdb-pid causes a horrible crash.
In gdb inside UML, putting a breakpoint on a int 0x80 causes the system
call not to happen.
Breakpoints don't work in the trivial statically linked getpid program.
PTRACE_SINGLESTEP appears not to be working.
Figure out why, when there are more telnet connections to UML consoles
than there are consoles, when one is freed up, the next telnet connection
only wakes up when something else causes an interrupt.
Figure out where the occasional 'end!=nsectors' message is coming from.
Setting up ethertap on 2.2 causes uml_net to spit out an infinite
stream of -EBADF.
Configuring a device twice with uml_mconsole apparently succeeds and
leaves it unusable.
Configuring a disk to a non-existant file with uml_mconsole apparently
succeeds and leaves it unusable.
I built UML inside itself twice on hostfs, with the output going to a
log. The first failed, and when the second ran, the log filled with
garbage.
The xtime lock should disable interrupts.
Confirm fix
Someone needs to try booting UML as a diskless client via bootp and
make sure that it works.
I have a bug that says that diskless booting doesn't work because it's
impossible to assign an address to a network interface from the
command line. However, I've heard of someone booting diskless
successfully, so this needs to be cleared up.
I have an old bug that claims that UML can be crashed with a ping
flood. I haven't seen this happen recently, so if anyone else has,
you'd better let me know.
Larent Bonnaud claimed a while ago that upgrading a Debian filesystem
hung the kernel. I need to know if this is still a problem.
Get Rik van Riel to confirm that UML no longer loses
characters that he types at the console. Rik says he hasn't seen it
in a while, mistral confirms.
If you get UML to crash, hang, or otherwise misbehave, you should
report this on one of the project mailing lists, either the
developer list - user-mode-linux-devel at lists dot sourceforge dot
net (subscription info) or the user list -
user-mode-linux-user at lists dot sourceforge dot net
(subscription info). When you do, it is
likely that I will want more information. So, it would be helpful to
read the stuff below, do whatever is applicable in your case, and
report the results to the list.
For any diagnosis, you're going to need to build a debugging kernel.
The binaries from this site aren't debuggable. If you haven't done
this before, read about
compiling and
debugging UML first.
Case 1 : Normal kernel panics
The most common case is for a normal thread to panic. To debug this,
you will need to run it under the debugger (add 'debug' to the command
line). An xterm will start up with gdb running inside it. Continue
it when it stops in start_kernel and make it crash. Now ^C gdb and
'bt'. I'm going to want to see the resulting stack trace.
If the panic was a "Kernel mode fault", then there will be a segv
frame on the stack and I'm going to want some more information. The
stack might look something like this:
(UML gdb) backtrace
#0 0x1009bf76 in __sigprocmask (how=1, set=0x5f347940, oset=0x0)
at ../sysdeps/unix/sysv/linux/sigprocmask.c:49
#1 0x10091411 in change_sig (signal=10, on=1) at process.c:218
#2 0x10094785 in timer_handler (sig=26) at time_kern.c:32
#3 0x1009bf38 in __restore ()
at ../sysdeps/unix/sysv/linux/i386/sigaction.c:125
#4 0x1009534c in segv (address=8, ip=268849158, is_write=2, is_user=0)
at trap_kern.c:66
#5 0x10095c04 in segv_handler (sig=11) at trap_user.c:285
#6 0x1009bf38 in __restore ()
I'm going to want to see the symbol and line information for the value
of ip in the segv frame. In this case, you would do the following:
(UML gdb) i sym 268849158
and
(UML gdb) i line *268849158
The reason for this is the __restore frame right above the
segv_handler frame is hiding the frame that actually segfaulted. So,
I have to get that information from the faulting ip.
Case 2 : Tracing thread panics
The less common and more painful case is when the tracing thread
panics. In this case, the kernel debugger will be useless because it
needs a healthy tracing thread in order to work. The first thing to
do is get a backtrace from the tracing thread. This is done by
figuring out what its pid is, firing up gdb, and attaching it to that
pid. You can figure out the tracing thread pid by looking at the
first line of the console output, which will look like this:
tracing thread pid = 15851
or by running ps on the host and finding the line that looks like
this:
If the panic was 'segfault in signals', then follow the instructions
above for collecting information about the location of the seg fault.
If the tracing thread flaked out all by itself, then send that
backtrace in and wait for our crack debugging team to fix the
problem.
Case 3 : Tracing thread panics caused by other threads
However, there are cases where the misbehavior of another
thread caused the problem. The most common panic of this type is:
wait_for_stop failed to wait for pid to stop with signal number
In this case, you'll need to get a backtrace from the process
mentioned in the panic, which is complicated by the fact that the
kernel debugger is defunct and without some fancy footwork, another
gdb can't attach to it. So, this is how the fancy footwork goes:
In a shell:
host% kill -STOP pid
Run gdb on the tracing thread as described in case 2 and do:
(host gdb) call detach(pid)
If you get a segfault, do it again. It always works the second
time.
Detach from the tracing thread and attach to that other thread:
(host gdb) detach
(host gdb) attach pid
If gdb hangs when attaching to that process, go back to a shell and
do:
host%
kill -CONT pid
And then get the backtrace:
(host gdb) backtrace
Case 4 : Hangs
Hangs seem to be fairly rare, but they sometimes happen. When a hang
happens, we need a backtrace from the offending process. Run the
kernel debugger as described in case 1 and get a backtrace. If the
current process is not the idle thread, then send in the backtrace.
You can tell that it's the idle thread if the stack looks like this:
#0 0x100b1401 in __libc_nanosleep ()
#1 0x100a2885 in idle_sleep (secs=10) at time.c:122
#2 0x100a546f in do_idle () at process_kern.c:445
#3 0x100a5508 in cpu_idle () at process_kern.c:471
#4 0x100ec18f in start_kernel () at init/main.c:592
#5 0x100a3e10 in start_kernel_proc (unused=0x0) at um_arch.c:71
#6 0x100a383f in signal_tramp (arg=0x100a3dd8) at trap_user.c:50
If this is the case, then some other process is at fault, and went to
sleep when it shouldn't have. Run ps on the host and figure out which
process should not have gone to sleep and stayed asleep. Then attach
to it with gdb and get a backtrace as described in case 3.
As an option, UML has the ability to log all data going through UML
terminals out to the host. This is primarily useful for
honeypots, although
other security-related applications might find it useful as well.
The first step in using this is to configure it in. It's enabled with
CONFIG_TTY_LOG ('Enable tty logging' under 'Character Devices' in the
UML configuration). With this done, UML will automatically log all
sessions to the host.
The interval between opening a UML terminal device and closing it is
considered to be a session. By default, each session is logged to a
separate file in the current directory on the host. The file name is
constructed from the current time. There will be a lot of them
created during the boot process because each step of bringing the
system up opens and closes /dev/console, which makes each line of boot
output look like a separate session. Every login will recognized as a
session, so those will each appear in its own file. If the user
allocates another tty (with su, screen, or some similar tool), that
will open a new session, which will get a separate log file on the
host.
Customization
There a couple of ways of changing this behavior. To have the log
files put in a different directory, use the tty_log_dir switch on the
UML command line:
tty_log_dir=dir
The one file per session scheme is the simplest possible way to do
logging, but it's limited and may not be suitable for everyone. The
main problem is that this definition of session isn't exactly the same
as the common notion, which is everything that appears on the user's
screen. Utilities which allocate new pseudo-terminals, such as su and
screen, will cause new log files to appear, and it won't necessarily
be obvious how to splice that into the log of the parent session in
order to reconstruct the text seen by the user.
The solution to this problem is to write the logging information out
as a single stream of data which contains information about the device
that's being used. This is done by specifying the tty_log_fd option
on the UML command line:
tty_log_fd=3 3>tty_log_file
This causes UML's file descriptor 3 to be opened to tty_log_file, and
the logging data will be written to it.
The logging data is a stream of fixed length records with optional
variable length data following.The records have the
following form:
struct tty_log_buf {
int what;
unsigned long tty;
int len;
int direction;
unsigned long sec;
unsigned long usec;
};
The 'tty' field is an integer to be used as a unique identifier of the
tty. It is actually the address within UML of the tty_struct, but
outside UML, it is used as an opaque identifier.
The 'len' field says how much data follows the record. It will be
non-zero for TTY_LOG_WRITE and TTY_LOG_OPEN (in UML 2.4.19-40 and
later) records, and zero for all others.
If the record type is TTY_LOG_WRITE, then the data that was written to
the tty immediately follows the tty_log_buf, and its 'len' field says
how much data there will be.
For TTY_LOG_OPEN records, the data length is sizeof(long) and the data
is the identifier of the tty that is active in the context of this
open. This is what allows character streams from different terminals
to be spliced back together to reproduce the stream of text that the
user actually saw. This identifier is the 'parent' tty, so the data
from the newly opened 'child' tty needs to be inserted at this point
into the parent's stream.
The 'direction' field says whether the data was being written to or
read from the terminal. It can have one of these values
#define TTY_READ 1
#define TTY_WRITE 2
The 'sec' and 'usec' fields are a timestamp, which is useful when
playing the log back with the original timings.
To fully use this functionality, you should use UML version 2.4.19-49
or later. tty logging in skas mode (skas mode is highly recommended for
all security-related applications) was broken because of a copy_user
bug. Also, TTY_LOG_OPEN wasn't including the parent tty in its data,
making session reconstruction impossible.
Log playback
jail/tty_log.pl in the utilities tarball contains a simple log
parser. It reads the records written to tty_log_fd, parses them, and
prints them out. It should be fairly easy to customize it to do
whatever session reconstruction you need.
Also in the utilities tarball is jail/playlog.pl, which is a more
user-friendly interface to the log. By default, it will play back the
session at its original speed if there is only one session in the
log. If there are multiple sessions, it will print out their ids and
exit. You must then rerun playlog, specifying which session you want
to see. In this case, the command line is
perl playlog.pl log-file [tty-id]
There are some switches which alter its behavior
-f - follows a live log, similar to 'tail -f'. This will show the
session live, in real time.
-n - dumps out the session without recreating the original timing
-a - prints out all data, rather than only tty output. This will
allow you to see things which didn't echo on the terminal, such as
passwords. The downside is that all other user input will be
doubled, since those characters are both tty input and tty output.
The virtual network laboratory is implemented on a 1 GHz Pentium III
system having 384 MBytes of RAM and a 20 GByte hard disk. The host
operating system is Redhat Linux 7.1 with a 512 MB swap partition. In
all, the virtual net consists of 20 virtual hosts sitting on 10
(sub)subnets connecting via one virtual router to our campus network
(and thence out to the Internet) as shown. The CPIT campus has been
assigned a class B address space which is subnetted into class C
address ranges (i.e. subnet mask=255.255.255.0). One of these /24
subnets has been allocated for this virtual lan server. On the
virtual network side of this lan the range is further /28 subnetted as
shown (i.e. netmask=255.255.255.240).
