pax_global_header 0000666 0000000 0000000 00000000064 14176035274 0014523 g ustar 00root root 0000000 0000000 52 comment=073d155a46fdf303ab8462f6e49665666e7f169f
hddemux-0.5/ 0000775 0000000 0000000 00000000000 14176035274 0013025 5 ustar 00root root 0000000 0000000 hddemux-0.5/.gitignore 0000664 0000000 0000000 00000000215 14176035274 0015013 0 ustar 00root root 0000000 0000000 *~
hddemux
hddemux.1
.refcache/
draft-dkg-dprive-demux-dns-http.xml
draft-dkg-dprive-demux-dns-http.html
draft-dkg-dprive-demux-dns-http.txt
hddemux-0.5/LICENSE 0000664 0000000 0000000 00000104517 14176035274 0014042 0 ustar 00root root 0000000 0000000 GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
Copyright (C)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
Copyright (C)
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
.
hddemux-0.5/Makefile 0000664 0000000 0000000 00000001755 14176035274 0014475 0 ustar 00root root 0000000 0000000 #!/usr/bin/make -f
CFLAGS += -D_GNU_SOURCE -g -O3
PKG_CONFIG ?= pkg-config
CFLAGS += $(shell $(PKG_CONFIG) --cflags libuv)
CFLAGS += $(shell $(PKG_CONFIG) --cflags libsystemd)
LDFLAGS += $(shell $(PKG_CONFIG) --libs libuv)
LDFLAGS += $(shell $(PKG_CONFIG) --libs libsystemd)
all: hddemux hddemux.1
check: hddemux
PATH=.:$$PATH ./testsuite
hddemux: hddemux.c
$(CC) $(CPPFLAGS) $(CFLAGS) $< -Wl,--as-needed $(LDFLAGS) -std=c11 -pedantic -Wall -Werror -o $@
hddemux.1: hddemux.1.md
pandoc -s -f markdown -t man -o $@ $<
draft-dkg-dprive-demux-dns-http.xml: draft-dkg-dprive-demux-dns-http.md
kramdown-rfc2629 < $< > $@
draft-dkg-dprive-demux-dns-http.html: draft-dkg-dprive-demux-dns-http.xml
xml2rfc $< --html
draft-dkg-dprive-demux-dns-http.txt: draft-dkg-dprive-demux-dns-http.xml
xml2rfc $< --text
clean:
rm -rf hddemux hddemux.1 \
.refcache/ \
draft-dkg-dprive-demux-dns-http.txt \
draft-dkg-dprive-demux-dns-http.html \
draft-dkg-dprive-demux-dns-http.xml
.PHONY: clean all check
hddemux-0.5/README.md 0000664 0000000 0000000 00000001601 14176035274 0014302 0 ustar 00root root 0000000 0000000 Given restrictive firewalls and intrusive network monitors, it can be
both privacy-preserving and connectivity-enhancing to be able to serve
both HTTPS (HTTP/1.x) and DNS-over-TLS from the same TCP port.
This project aims to document specifically how that can be done
safely, and to provide simple code to demonstrate the mechanism.
As of this writing, the code is deployed and in-use at the host
`dns.cmrg.net`, which offers both HTTPS and a full recursive resolver
(via DNS-over-TLS), both on port 443. Please see https://dns.cmrg.net
for more details of that service.
This project includes:
* `hddemux.c` -- a `libuv`-based HTTP/1.x and DNS multiplexing server
* `hddemux.socket` and `hddemux.service` -- systemd units to manage
`hddemux`
* `draft-dkg-dprive-demux-dns-http.md` -- an Internet Draft
documenting the rationale and the algorithm.
Patches and commentary welcome!
hddemux-0.5/TODO.md 0000664 0000000 0000000 00000001211 14176035274 0014107 0 ustar 00root root 0000000 0000000 * Cleanup outstanding idle connections more cleverly -- can we only do
this cleanup when under memory pressure? is there a better way to
choose which connections to kill? IDLE_TIMEOUT is a pretty clumsy
knob compared to ideal memory management practices.
* Enable unix-domain sockets in the HTTP_TARGET and DNS_TARGET
backends -- no reason that these need to be TCP addresses.
* Are there other client-speaks-first streaming protocols that we can
demux in addition to HTTP and DNS?
* Consider making hddemux work on non-systemd systems (e.g. pass a set
of listener arguments on the CLI in addition to looking for
SD_LISTEN_FDS)
hddemux-0.5/draft-dkg-dprive-demux-dns-http.md 0000664 0000000 0000000 00000070206 14176035274 0021365 0 ustar 00root root 0000000 0000000 ---
title: Demultiplexing Streamed DNS from HTTP/1.x
docname: draft-dkg-dprive-demux-dns-http-03
date: 2017-05-17
category: info
ipr: trust200902
area: int
workgroup: dprive
keyword: Internet-Draft
updates: 1035, 7230
stand_alone: yes
pi: [toc, sortrefs, symrefs]
author:
-
ins: D. K. Gillmor
name: Daniel Kahn Gillmor
org: American Civil Liberties Union
street: 125 Broad St.
city: New York, NY
code: 10004
country: USA
abbrev: ACLU
email: dkg@fifthhorseman.net
informative:
RFC3007:
RFC5246:
RFC6840:
RFC6895:
RFC7301:
RFC7540:
RFC7830:
RFC7858:
I-D.ietf-dnsop-dns-wireformat-http:
I-D.hoffman-dns-over-https:
HAPROXY:
target: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
title: The Proxy protocol
author:
name: Willy Tarreau
ins: W. Tarreau
org: HAProxy Technologies
date: 2017-03-10
normative:
RFC1035:
RFC1945:
RFC2119:
RFC2136:
RFC5234:
RFC7230:
--- abstract
DNS over TCP and HTTP/1.x are both stream-oriented,
client-speaks-first protocols. They can both be run over a
stream-based security protocol like TLS. A server accepting a
stream-based client can distinguish between a valid stream of DNS
queries and valid stream of HTTP/1.x requests by simple observation of
the first few octets sent by the client. This can be done without any
external demultiplexing mechanism like TCP port number or ALPN.
Implicit multiplexing of the two protocols over a single listening
port can be useful for obscuring the presence of DNS queries from a
network observer, which makes it relevant for DNS privacy.
Widespread adoption of the described approach could constrain
evolution of the stream-based variants of both DNS (RFC1035) and
HTTP/1.x (RFC7230) by ossifying existing distinct bit patterns found
in early octets sent by the client. However, this draft explicitly
rules out multiplexing in this form with HTTP/2, so it should place no
constraints on the evolution of HTTP/2 or any higher version of HTTP.
--- middle
Introduction
============
DNS and HTTP/1.x are both client-speaks-first protocols capable of
running over stream-based transport like TCP, or as the payload of a
typical TLS {{RFC5246}} session.
There are some contexts where it is useful for a server to be able to
decide what protocol is used by an incoming TCP stream, to choose
dynamically between DNS and HTTP/1.x on the basis of the stream itself
(rather than a port designation or other explicit demultiplexing).
For example, a TLS terminator listening on port 443 and receiving
either no ALPN token at all, or the "http/1.1" ALPN token might be
willing to serve DNS-over-TLS {{RFC7858}} as well as HTTPS.
A simple demultiplexing server should do this demuxing based on the
first few bytes sent by the client on a given stream; once a choice
has been established, the rest of the stream is committed to one or
the other interpretation.
This document provides proof that a demultiplexer can robustly
distinguish HTTP/1.x from DNS on the basis of the content of the
first few bytes of the client's stream alone.
A DNS client that knows it is talking to a server which is this
position (e.g. trying to do DNS-over-TLS on TCP port 443 with no ALPN
token, used traditionally only for HTTPS) might also want to be aware
of network traffic patterns that could confuse such a server. This
document presents explicit mitigations that such a DNS client MAY
decide to use.
This document limits its discussion to HTTP/1.x over TCP or TLS or
some other classical stream-based protocol (it excludes HTTP over
QUIC, for example, and HTTP/2 {{RFC7540}} or later). Likewise, it
considers only the TCP variant of DNS (and excludes DNS over UDP or
any other datagram transport).
Terminology
-----------
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
{{RFC2119}}.
Scoping
=======
Distinguish only at the start of a stream
-----------------------------------------
A server which attempts to distinguish DNS queries from HTTP/1.x requests
individually might consider using these guidelines in the middle of a
running stream (e.g. at natural boundaries, like the end of an HTTP/1.1
request, or after a DNS message), but this document focuses
specifically on a heuristic choice for the whole stream, based on the
initial few octets sent by the client.
While it's tempting to consider distinguishing at multiple points in
the stream, the complexities of determining the specific end of an
HTTP/1.x request body and handling HTTP/1.x error cases make this more
difficult to implement on the side of a DNS client configured to talk
to such a server. Interleaving the responses themselves on a stream
with multiple data elements is also challenging. So do not use this
technique anywhere but at the beginning of a stream!
If being able to interleave DNS queries with HTTP requests on a single
stream is desired, a strategy like {{I-D.hoffman-dns-over-https}} or
{{I-D.ietf-dnsop-dns-wireformat-http}} is recommended instead.
HTTP/2 is not always client-speaks-first
----------------------------------------
While this demultiplexing technique functions for HTTP/1.0 and
HTTP/1.1, it does not work for HTTP/2 {{RFC7540}} because HTTP/2 is
not guaranteed to be a client-speaks-first protocol. In particular,
many HTTP/2 servers prefer to send a SETTINGS frame immediately
without waiting for data from the client, if they already know they're
speaking HTTP/2. In the event that HTTP/2 is to be transported over
TLS, the ALPN token negotiated in the TLS handshake is "h2", which
allows the server to know as soon as the handshake is complete that it
can start pushing data to the client.
A standard DNS-over-TLS client connecting to a server that might be
multiplexing DNS with HTTP on the same listener MUST NOT indicate an
intent to speak HTTP/2 that could prompt this unsolicited first flight
from the server. Concretely, a DNS client connecting over TLS on TCP
port 443 expecting to speak standard DNS-over-TLS {{RFC7858}} MUST NOT
offer or accept the "h2" ALPN token.
If use of DNS in the same channel as HTTP/2 is deisred, a strategy
like {{I-D.hoffman-dns-over-https}} is recommended instead.
Avoid multiplexing in the clear
-------------------------------
The widespread deployment of transparent HTTP/1.x proxies makes it
likely that any attempt to do this kind of multiplexing/demultiplexing
on a cleartext channel that normally carries HTTP/1.x (e.g. TCP port
80) will fail or trigger other "interesting" behaviors. The approach
described in this draft should be done only in channels sufficiently
obscured that a transparent proxy would not try to interpret the
resultant stream.
Avoid mixing with other demultiplexing
--------------------------------------
Some other (non-IETF) systems (e.g. {{HAPROXY}}) take a similar
approach with multiplexing data on top of HTTP/1.x by taking advantage
of bitpatterns that are presumed to not be present in normal HTTP/1.x
requests.
Use of the approach described in this draft in conjunction with these
other approaches is not advisable. Doing so safely would require
explicit and detailed review of all three (or more) protocols
involved.
Heavily-restricted network environments
---------------------------------------
Some network environments are so tightly constrained that outbound
connections on standard TCP ports are not accessible. In some of
these environments, an explicit HTTP proxy is available, and clients
must use the HTTP CONNECT pseudo-method to make https connections.
While this multiplexing approach can be used in such a restrictive
environment, it would be necessary to teach the DNS client how to talk
to (and through) the HTTP proxy. These details are out of scope for
this document. A DNS client capable of this additional layer of
complexity may prefer to pursue a strategy like
{{I-D.hoffman-dns-over-https}} instead.
Why not ALPN?
-------------
If this is done over TLS, a natural question is whether the client
should simply indicate its preferred protocol in the TLS handshake's
ALPN {{RFC7301}} extension (e.g. with some new ALPN token "dns").
However, ALPN tokens requested by the client are visible to a network
observer (and the ALPN token selected by the server is visible to a
network observer in TLS 1.2 and earlier), so a network controller
attempting to confine the user's DNS traffic to a limited set of
servers could use the ALPN extension as a signal to block DNS-specific
streams.
Another alternative could be an ALPN token that indicates
potentially-multiplexed traffic (e.g. "http/1.1-or-dns"). This has a
comparable problem when confronted with a network adversary that
intends to penalize or hamper DNS-over-TLS. Existing HTTP clients
will not send this token, and even if some start to offer it, it will
provide less cover for DNS-over-TLS clients.
Overview of initial octets
==========================
DNS stream initial octets
-------------------------
{{RFC1035}} section 4.2.2 ("TCP Usage") shows that every stream-based
DNS connection starts with a DNS message, preceded with a 2-octet
message length field:
The message is prefixed with a two byte length field which gives
the message length, excluding the two byte length field.
{{RFC6895}} section 2 represents the DNS message header section,
which is the first part of the DNS message on the wire (after the
message length).
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ID |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|QR| OpCode |AA|TC|RD|RA| Z|AD|CD| RCODE |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| QDCOUNT/ZOCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ANCOUNT/PRCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| NSCOUNT/UPCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ARCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
So in a DNS over TCP stream, the interpretation of the initial 14
octets are fixed based on information about the first query sent on the
stream:
* 0,1: length of initial DNS message
* 2,3: DNS Transaction ID
* 4,5: DNS opcode, flags, and response code
* 6,7: Question count (or Zone count in UPDATE)
* 8,9: Answer count (or Prerequisite count in UPDATE)
* 10,11: Authority count (or Update count in UPDATE)
* 12,13: Additional RR count
All DNS streams sent over TCP start with at least these 14 octets.
HTTP/1.x initial octets
-------------------
In an HTTP stream before HTTP/2, the first octets sent from the client
are either the so-called `Simple-Request` (for HTTP/0.9) or the
`Request-Line` (for HTTP/1.0 and HTTP/1.1). The data in this initial
stream has variable characteristics.
Most servers may wish to ignore the oldest of these, HTTP/0.9.
### HTTP/0.9
{{RFC1945}} section 4.1 says that HTTP/0.9 queries (that is, HTTP
queries from before HTTP/1.0 was formalized) use this form:
Simple-Request = "GET" SP Request-URI CRLF
Note that HTTP/0.9 clients send this string and only this string,
nothing else (no request body, no subsequent requests). The
`Request-URI` token is guaranteed to start with a printable ASCII
character, and cannot contain any members of the CTL class (values
0x00 through 0x1F) but due to loose early specifications, it might
sometimes contain high-valued octets (those with the most-significant
bit set -- 0x80 or above).
So the first 5 octets are all constrained to be no less than 0x20 (SP)
and no more than 0x7F (DEL), and all subsequent octets sent from the
client have a value at least 0x0A (LF).
The shortest possible HTTP/0.9 client request is:
char: G E T SP / CR LF
index: 0 1 2 3 4 5 6
The lowest possible HTTP/0.9 client request (sorted ASCIIbetically) is:
char: G E T SP + : CR LF
index: 0 1 2 3 4 5 6 7
### HTTP/1.0 and HTTP/1.1
The request line format for HTTP/1.1 matches that of HTTP/1.0
(HTTP/1.1 adds protocol features like pipelining, but doesn't change
the request form itself). But unlike HTTP/0.9, the initial verb (the
`method`) can vary.
{{RFC7230}} section 3.1.1 says that the first line of an HTTP/1.1
request is:
request-line = method SP request-target SP HTTP-version CRLF
method = token
and {{RFC7230}} section 3.2.6 says:
token = 1*tchar
tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*"
/ "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
/ DIGIT / ALPHA
; any VCHAR, except delimiters
and VCHAR is defined in {{RFC5234}} appendix B.1 as:
VCHAR = %x21-7E
`request-target` itself cannot contain 0x20 (SP) or any CTL
characters, or any characters above the US-ASCII range (> 0x7F).
And the `HTTP-version` token is either the literal string "HTTP/1.0"
or the literal string "HTTP/1.1", both of which are constrained to the
same printable-ASCII range.
The ASCIIbetically-lowest shortest possible HTTP/1.0 or HTTP/1.1 request is:
char: ! SP / SP H T T P / 1 . 0 CR LF CR LF
index: 0 1 2 3 4 5 6 7 8 9 a b c d e f
In any case, no HTTP/1.0 or HTTP/1.1 request line can include any
values lower than 0x0A (LF) or greater than 0x7F (DEL) in the first 16
octets.
However, {{RFC7230}} section 3.1.1 also says:
In the interest of robustness, a server that is expecting to receive
and parse a request-line SHOULD ignore at least one empty line (CRLF)
received prior to the request-line.
So we should also consider accepting an arbitrary number of repeated
CRLF sequences before the request-line as a potentially-valid HTTP
client behavior.
Specific octets
===============
The sections below examine likely values of specific octet positions
in the stream. All octet indexes are 0-based.
octets 0 and 1
--------------
Any DNS message less than 3338 octets sent as the initial query over
TCP can be reliably distinguished from any version of HTTP/1.x by the
first two octets of the TCP stream alone.
3338 is 0x0D0A, or the ASCII string CRLF, which some HTTP/1.x clients
might send before an initial request. No HTTP/1.x client can
legitimately send anything lower than this.
Most DNS queries are easily within this range automatically.
octets 2 and 3
--------------
In a DNS stream, octets 2 and 3 represent the client-chosen message
ID. The message ID is used to bind messages with responses. Over
connectionless transports like UDP, this is an important anti-spoofing
measure, as well as a distinguishing measure for clients reusing the
same UDP port for multiple outstanding queries. Standard DNS clients
already explicitly randomize this value.
For the connection-oriented streaming DNS discussed here, the
anti-spoofing characteristics are not relevant (the connection itself
provides anti-spoofing), so the client is free to choose arbitrary
values.
With a standard DNS client which fully-randomizes these values, only
25% of generated queries will have the high bits of both octets set to
0. 100% of all HTTP/1.x requests will have the high bits of both of
these octets cleared. Similarly, some small percentage of
randomly-generated DNS queries will have values here lower than 0x0A,
while no HTTP/1.x clients will ever send these low values.
octet 4
-------
In a DNS stream, octet 4 combines several fields:
0 1 2 3 4 5 6 7
+--+--+--+--+--+--+--+--+
|QR| Opcode |AA|TC|RD|
+--+--+--+--+--+--+--+--+
In a standard DNS query sent over a streaming interface, QR, Opcode,
AA, and TC are all set to 0. The least-significant bit (RD --
Recursion Desired) is set when a packet is sent from a stub to a
recursive resolver. The value of such an octet is 0x01. This value
never occurs in octet 4 of a legitimate HTTP/1.x client.
But under DNS UPDATE ({{RFC2136}}, Opcode is set to 5 and all the
option bits are cleared, which means this value would have 0x40 (ASCII
'@'), which could legitimately occur in some HTTP/1.x requests at this
position.
octet 5
-------
In a DNS stream, octet 5 also combines several fields:
0 1 2 3 4 5 6 7
+--+--+--+--+--+--+--+--+
|RA| Z|AD|CD| RCODE |
+--+--+--+--+--+--+--+--+
In some DNS messages sent from a client, all these bits are 0.
However, section 5.7 of {{RFC6840}} suggests that queries may wish to
set the AD bit to indicate a desire to learn from a validating
resolver whether the resolver considers the contents to be Authentic
Data.
{{RFC6840}} also suggests that:
validating resolvers SHOULD set the CD bit on every upstream query.
So many queries, particularly from DNSSEC-validating DNS clients, are
likely to set bits 2 and 3, resulting in a value 0x30 (ASCII '0').
This is usually a legitimate value for octet 5 in an HTTP/1.x request.
octets 6 and 7
--------------
In DNS, octets 6 and 7 represent the query count. Most DNS clients
will send one query at a time, which makes this value 0x0001. As long
as the number of initial queries does not exceed 0x0A0A (2570), then
at least one of these octets will have a value less than 0x0A. No
HTTP/1.x client sends an octet less than 0x0A in positions 6 or 7.
In DNS UPDATE, octets 6 and 7 represent the zone count. Entries in
the Zone section of the DNS UPDATE message are structured identically
to entries in the Query section of a standard DNS message.
octets 8 through 11
-------------------
In streaming DNS, octets 8 through 11 represent answer counts and
authority counts in normal DNS queries, or Prerequisite and Update
counts in DNS UPDATE. Standard DNS queries will set them both 0. DNS
UPDATE queries are likely to include some records in these sections,
so they won't be all zero, but as long as no more than 2570
Prerequisite records and no more than 2570 Update records are sent, at
least one octet will have value less than 0x0A. But no HTTP/1.x
client sends an octet less than 0x0A in these positions.
octets 12 and 13
----------------
In streaming DNS, octets 12 and 13 represent the number of Additional
RRs. When a DNS query is sent with EDNS(0), the OPT RR is accounted
for here. So this is often either 0x0000 or 0x0001. In a Secure DNS
UPDATE {{RFC3007}}, the SIG(0) or TSIG record is also found in this
section, which could increase the values of these octets to 0x0002.
No HTTP/1.x client will send octets with these low values at these
positions.
Combinations of octets
======================
In a DNS message, each Question in the Question section (or Zone in
the Zone section for DNS UPDATE) is at least 5 octets (1 octet for
zero-length QNAME + 2 octets for QTYPE + 2 octets for QCLASS), and
each RR (in the Answer, Authority, and Additional sections for normal
DNS queries; or in the Prerequisite, Update, and Additional sections
for DNS UPDATE) is at least 11 octets. And the header itself is 12
octets.
So we know that for a valid DNS stream, the first message has a size
of at least:
min_first_msg_size = 12 + 5 * (256*o[6] + o[7]) +
11 * (256*(o[8] + o[10] + o[12]) +
o[9] + o[11] + o[13])
It's possible to compare this value with the expected first query size:
first_msg_size = 256 * o[0] + o[1]
if `first_query_size` is less than `min_first_query_size` we can be
confident that the stream is not DNS.
### Proof: a valid DNS message cannot be an HTTP/1.x query
For any a valid, stream-based DNS message:
* If there are fewer than 0x0A00 Questions then octet 6 < 0x0A.
* If there are fewer than 0x0A00 Answer RRs, then octet 8 < 0x0A.
* If there are fewer than 0x0A00 Authority RRs, then octet 10 < 0x0A.
* If there are fewer than 0x0A00 Additional RRs, then octet 12 < 0x0A.
If any of these four inequalities hold, then the packet is clearly
DNS, not HTTP/1.x.
if none of them hold, then there are at least 0x0A00 (2560) Questions
and 3*2560 == 7680 RRs. But:
12 + 5*2560 + 11*7680 == 97292
So the smallest possible DNS message where none of these four
inequalities hold is 97292 octets. But a DNS message is limited in
size to 65535 octets.
Therefore at least one of these inequalities holds, and one of the
first 14 octets of a DNS steam is < 0x0A.
But in a standard HTTP/1.x request, none of the first 14 octets can
have a value < 0x0A, so a valid DNS message cannot be mistaken for an
HTTP/1.x request.
Guidance for Demultiplexing Servers
===================================
Upon receiving a connection stream that might be either DNS or
HTTP/1.x, a server can inspect the initial octets of the stream to
decide where to send it.
Without supporting HTTP/0.9
---------------------------
A server that doesn't care about HTTP/0.9 can simply wait for the
first 14 octets of the client's request to come in. Then the
algorithm is:
bytestream = read_from_client(14)
for x in bytestream:
if (x < 0x0A) or (x > 0x7F):
return `DNS`
return `HTTP`
Supporting archaic HTTP/0.9 clients
-----------------------------------
A server that decides to try to support HTTP/0.9 clients has a
slightly more challenging task, since some of them may send fewer
octets than the initial DNS message, and the server shouldn't block
waiting for data that will never come.
bytestream = read_from_client(5)
for x in bytestream[0:5]
if (x < 0x0A) or (x > 0x7F):
return `DNS`
if (bytestream[0:4] != 'GET '): # not HTTP/0.9
bytestream += read_from_client(9)
for x in bytestream[5:14]:
if (x < 0x0A) or (x > 0x7f):
return `DNS`
return `HTTP`
else: # maybe HTTP/0.9
seen_sp = False
seen_high = False
while (len(bytestream) < 14):
if (seen_sp and seen_high):
return `DNS`
x = read_from_client(1)
bytestream += x
if (x > 0x7F):
seen_high = True
elif (x < 0x0A):
return `DNS`
elif (x == 0x20):
seen_sp = True # SP found before CRLF, not HTTP/0.9
elif (x == 0x0A):
return `HTTP`
return `HTTP`
Note that if read_from_client() ever fails to read the number of
requested bytes (e.g. because of EOF), then the stream is neither
valid HTTP nor valid DNS, and can be discarded.
Signaling demultiplexing capacity
---------------------------------
This document assumes that clients can learn out-of-band which
listening service they can connect to. For example, the administrator
of a machine can configure a local forwarding stub resolver to use
DNS-over-TLS on port 443 of some specific server. This explicit
configuration carries with it some level of trust -- the client is
choosing to trust the configured server with its DNS queries.
In some circumstances, it might be useful for a listener to signal to
a client that it is willing and capable of handling both DNS and
HTTP/1.x traffic. While such signalling could be useful for dynamic
discovery, it opens questions of trust (which servers should the
client be willing to rely on for DNS resolution?) and is out-of-scope
for this draft.
Guidance for DNS clients
========================
Consider a DNS client that connects to a server that might be
interested in answering HTTP/1.x requests on the same address/port (or
other channel identifier). The client wants to send traffic that is
unambiguously DNS traffic to make it easy for the server to
distinguish it from inbound HTTP/1.x requests. Fortunately, this is
trivial to do. In fact, any sensibly-implemented DNS-over-TLS client
can use this approach without modification, just by adjusting the port
number of the upstream recursive resolver from 853 to 443.
Such a client should follow these guidelines:
* Send the DNS message size (a 16-bit integer) together in the same
packet with the full header of the first DNS message so that the
recipient can review as much as possible of the frame at once.
This is a best practice for efficient stream-based DNS anyway.
If the client is concerned about stream fragmentation that it cannot
control, and it is talking to a server that might be expecting
HTTP/0.9 clients, then the server might not be willing to wait for the
full initial 14 octets to make a decision.
Note that this fragmentation is not a concern for streams wrapped in
TLS when using modern AEAD ciphersuites. In this case, the client
gets to choose the size of the plaintext record, which is either
recovered by the server in full (unfragmented) or the connection
fails.
If the client does not have such a guarantee from the transport, it
MAY also take one of the following mitigating actions relating to the
first DNS message it sends in the stream \[explanation of what the
server gets to see in the fragmented stream case are in square
brackets after each mitigation]:
* Ensure the first message is marked as a query (QR = 0), and it
uses opcode 0 ("Standard Query"). \[bytestream\[4] < 0x08]
* Ensure that the first message has RA = 0, Z = 0, and RCODE = 0.
\[bytestream\[5] == 0x00]
* Ensure that the high bit of the first octet of the message ID of
the first message is set. \[bytestream\[2] > 0x7F]
* Send an initial short Server Status DNS message ahead of the
otherwise intended initial DNS message. \[bytestream\[0] == 0x00]
* Use the EDNS(0) padding option {{RFC7830}} to pad the first
message to a multiple of 256 octets. \[bytestream\[1] == 0x00]
Interpreting failure
--------------------
FIXME: A DNS client that does not already know that a server is
willing to carry both types of traffic SHOULD expect a transport
connection failure of some sort. Can we say something specific about
what it should expect?
Guidance for HTTP clients
=========================
HTTP clients SHOULD NOT send HTTP/0.9 requests, since modern HTTP
servers are not required to support HTTP/0.9. Sending an HTTP/1.0
request (or any later version) is sufficient for a server to be able
to distinguish the two protocols.
Security Considerations
=======================
FIXME: Clients should locally validate DNSSEC (servers may still be
able to omit some records)
FIXME: if widely deployed, consider amplification for DDoS against
authoritative servers?
FIXME: consider DNSSEC transparency
FIXME: consider TLS session resumption -- this counts as a new stream
boundary, so the multiplexing decision need not persist across
resumption.
FIXME: consider 0-RTT
FIXME: consider X.509 cert validation
FIXME: what other security considerations should clients take?
FIXME: what other security considerations should servers take?
Privacy Considerations
======================
FIXME: DNS queries and HTTP requests can reveal potentially sensitive
information about the sender.
FIXME: consider DNS and HTTP traffic analysis -- how should requests
or responses be padded, aggregated, or delayed given that streams are
multiplexed?
FIXME: any other privacy considerations?
IANA Considerations
===================
This document does not ask IANA to make any changes to existing
registries.
However, it does update the DNS and HTTP specifications, to reflect
the fact that services using this demultiplexing technique may be
constrained in adoption of future versions of either stream-based DNS
or HTTP/1.x if those future versions modify either protocol in a way
that breaks with the distinctions documented here.
In particular, this draft assumes that all future stream-based
versions of HTTP/1.x should have the following properties:
* the client will speak first
* the client will send at least 14 octets before expecting a response
from the server.
* none of those first 14 octets will be below 0x0A (LF) or above 0x7F
(DEL).
Future extensions to stream-based DNS or HTTP/1.x should take this
demultiplexing technique into consideration.
Document Considerations
=======================
\[ RFC Editor: please remove this section before publication ]
This document is currently edited as markdown. Minor editorial
changes can be suggested via merge requests at
https://gitlab.com/dkg/hddemux or by e-mail to the author. Please
direct all significant commentary to the public IETF DNS Privacy
mailing list: dns-privacy@ietf.org or to the IETF HTTP WG mailing
list: ietf-http-wg@w3.org
hddemux-0.5/hddemux.1.md 0000664 0000000 0000000 00000006002 14176035274 0015142 0 ustar 00root root 0000000 0000000 ---
title: HDDEMUX
section: 1
author: Daniel Kahn Gillmor
date: 2017 May
---
NAME
====
hddemux - demultiplexes incoming TCP connections between HTTP and DNS
SYNOPSIS
========
hddemux
DESCRIPTION
===========
hddemux takes a set of listening stream-based file descriptors (see
sd_listen_fds(3)) and accepts new connections on them.
When a new connection comes in, it decides from the first few octets
whether the connection is HTTP/1.x or DNS. If it thinks it's
HTTP/1.x, it splices the connection to the HTTP_TARGET. If it thinks
it's DNS, it splices the connection to the DNS_TARGET.
ENVIRONMENT VARIABLES
=====================
HTTP_TARGET and DNS_TARGET should be either a DNS name or an IP address,
optionally followed by a colon and a port number. If either variable
is unset, it will default to "localhost".
If the port number is not specified, HTTP_TARGET defaults to TCP port
80, and DNS_TARGET defaults to TCP port 53.
If HDDEMUX_DEBUG is set to a non-empty string, then it will send a lot
of verbose debugging info to stderr.
IDLE_TIMEOUT is used to set the number of milliseconds that hddemux
will permit a session to be in an open state with no traffic passing
before it tries to close both sides. The default is 600000 (10
minutes). Set this variable to 0 to never try to do this kind of
cleanup.
SIGNALS
=======
When hddemux receives SIGUSR1, it dumps an overview of the current
state of the demuxer (configuration, outstanding established streams,
etc) to stderr.
EXAMPLES
========
For systemd, you need a .socket unit file:
#/lib/systemd/hddemux.socket
[Unit]
Description=HTTP/1.x and DNS demuxer socket
Documentation=man:hddemux(1)
[Socket]
ListenStream=/run/hddemux/socket
[Install]
WantedBy=sockets.target
and a .service unit file:
#/lib/systemd/hddemux.service
[Unit]
Description=HTTP/1.x and DNS demuxer service
Documentation=man:hddemux(1)
Requires=hddemux.socket
[Service]
Type=notify
ExecStart=/usr/bin/hddemux
WorkingDirectory=/run/hddemux/workdir
User=hddemux
Group=hddemux
[Install]
Also=hddemux.socket
Configuration can be done by overriding the .service file (e.g. with
Service.Environment= entries). See the "edit" documentation in
systemctl(1) and the "Overriding vendor settings" section of
systemd.unit(5).
CONSTRAINTS
===========
hddemux is designed to demultiplex HTTP/1.x from stream-based DNS.
Trying to demultiplex other protocols (including HTTP/2 or later) is
not advised. Please see draft-dkg-dprive-demux-dns-http for more
information and analysis.
WARNING
=======
Note that this effectively acts as a stream redirector once the
client's first flight has been processed. It does this with no
attempt to defend against stream redirection loops, so be careful not
to redirect it to itself or it will eat all of your memory in infinite
recursion.
SEE ALSO
========
sd_listen_fds(3), systemctl(1), systemd.unit(5),
https://datatracker.ietf.org/doc/draft-dkg-dprive-demux-dns-http/
hddemux-0.5/hddemux.c 0000664 0000000 0000000 00000067542 14176035274 0014645 0 ustar 00root root 0000000 0000000 #include
#include
#include
#include
#include
#include
#include
#include
#include
/* library includes */
#include
#include
#include
#define INTERESTING_OCTETS 14
#define DEFAULT_IDLE_TIMEOUT (10*60*1000)
enum protocol {
DEMUX_HTTP = 0,
DEMUX_DNS = 1,
DEMUX_UNKNOWN = 2,
};
const char *protocol_names[] = { "HTTP", "DNS", "UNKNNOWN" };
/* see README.md for justification of the heuristics below: */
static inline enum protocol surmise_protocol(unsigned char *bytestream, size_t len) {
if (len < 5)
return DEMUX_UNKNOWN;
int ix;
bool all_interesting = len >= INTERESTING_OCTETS;
for (ix = 0; ix < 5; ix++)
if ((bytestream[ix] < 0x0a) || (bytestream[ix] > 0x7f))
return DEMUX_DNS;
if (memcmp(bytestream, "GET ", 4) != 0) { /* not HTTP/0.9 */
for (ix = 5; ix < (all_interesting ? INTERESTING_OCTETS : len); ix++)
if ((bytestream[ix] < 0x0a) || (bytestream[ix] > 0x7f))
return DEMUX_DNS;
} else { /* might be HTTP/0.9 */
bool seen_sp = false;
bool seen_high = false;
for (ix = 5; ix < (all_interesting ? INTERESTING_OCTETS : len); ix++) {
if (seen_sp && seen_high)
return DEMUX_DNS;
if (bytestream[ix] > 0x7f)
seen_high = true;
else if (bytestream[ix] < 0x0a)
return DEMUX_DNS;
else if (bytestream[ix] == 0x20)
seen_sp = true;
else if (bytestream[ix] == 0x0a)
return DEMUX_HTTP;
}
}
if (all_interesting)
return DEMUX_HTTP;
else
return DEMUX_UNKNOWN;
}
struct demuxer;
/* managed streams, which will be sorted by last_touched */
struct streamstate {
int id;
struct {
uv_tcp_t stream;
size_t incoming;
unsigned char *staging;
size_t bytes_read;
size_t bytes_written;
uv_write_t writer;
uv_shutdown_t shutdown;
bool initialized;
bool eof_seen;
bool closed;
} x[2];
size_t first_read_size;
enum protocol surmised_protocol;
uv_connect_t connect;
bool released;
uint64_t last_touched;
uv_timer_t timer;
bool timer_closed;
};
int stream_splice(struct streamstate *s, uv_stream_t *from);
int stream_exchange(struct streamstate *s);
struct streamstate *new_streamstate(struct demuxer* demux);
void free_streamstate(struct streamstate *str);
void touch_streamstate(struct streamstate *st);
/* program state: */
struct demuxer {
struct addrinfo *http_targets, *dns_targets, *next_http, *next_dns;
uv_loop_t *loop;
uv_signal_t usr1_signal;
uv_signal_t term_signal;
int next_stream_id;
int active_streams;
bool debug;
uint64_t stream_timeout; /* in milliseconds */
uv_tcp_t *listeners;
int max_listeners;
int listener_count;
};
struct demuxer* new_demuxer(uv_loop_t *loop, int max_listeners);
void free_demuxer(struct demuxer* demux);
static void demuxer_rotate(struct demuxer *demuxer, enum protocol proto);
static const struct addrinfo* demuxer_getaddrinfo(const struct demuxer *demuxer, enum protocol proto);
static void demuxer_dump(struct demuxer *demuxer, FILE *f);
void demuxer_signal_usr1(uv_signal_t *sig, int signum);
void demuxer_signal_term(uv_signal_t *sig, int signum);
/* workflow steps */
void new_inbound(uv_stream_t* server, int status);
void first_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf);
void inbound_first_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf);
void stream_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf);
void outbound_connection(uv_connect_t *req, int status);
void write_complete(uv_write_t* write, int status);
void stream_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf);
void interesting_sent(uv_write_t* req, int status);
void after_shutdown(uv_shutdown_t* req, int status);
void close_streamstate_stream(uv_handle_t *stream);
void close_streamstate_timer(uv_handle_t *timer);
void streamstate_timeout(uv_timer_t *timer);
struct min_max_streamstate {
const struct streamstate *min, *max;
};
void get_min_max_streamstate(uv_handle_t *h, void *x);
/* *************** utilities *************** */
static void dump_addrinfo(FILE *f, const char *label, const struct addrinfo *a, bool is_current) {
int port = 0;
const char *fam;
char paddr[INET6_ADDRSTRLEN];
const void *ia = NULL;
if (a->ai_family == AF_INET6) {
struct sockaddr_in6* ip = (struct sockaddr_in6*)a->ai_addr;
port = ntohs(ip->sin6_port);
fam = "AF_INET6";
ia = &(ip->sin6_addr);
} else if (a->ai_family == AF_INET) {
struct sockaddr_in* ip = (struct sockaddr_in*)a->ai_addr;
port = ntohs(ip->sin_port);
fam = "AF_INET";
ia = &(ip->sin_addr);
} else {
fam = "UNKNOWN";
}
if (ia) {
if (inet_ntop(a->ai_family, ia, paddr, sizeof(paddr)) == NULL) {
int e = errno;
/* these shenanigans are all for the GNU variant of
strerror_r(3), not the XSI variant: */
const char * e2 = strerror_r(e, paddr+6, sizeof(paddr)-(6+1));
if (e2 != paddr+6)
strncpy(paddr+6, e2, sizeof(paddr)-(6+1));
memcpy(paddr, " ", 6);
paddr[sizeof(paddr)-1] = '\0';
}
} else {
strcpy(paddr, "");
}
fprintf(f, "%s%s: fam: %s(%d) socktype: %d, proto: %d, addr: %s, port: %d\n",
(is_current ? "[*]" : " "), label, fam,
a->ai_family, a->ai_socktype, a->ai_protocol,
paddr, port);
}
static int get_target_addrinfo(const char* envname, const char* defaultnode, const char *defaultservice, struct addrinfo**res, bool debug) {
char *n = getenv(envname);
const char *node = NULL, *service = NULL;
const struct addrinfo hints = {
.ai_flags = AF_UNSPEC,
.ai_socktype = SOCK_STREAM
};
if (n != NULL) {
char *colon = strrchr(n, ':');
char *bracket = strrchr(n, ']');
if (debug)
fprintf(stderr, "target for %s is %s\n", envname, n);
if (colon != NULL) {
if (bracket != NULL && (bracket > colon)) {
/* this was just a colon in a bracketed IPv6 address, not a
port delimiter */
} else {
*colon = '\0';
service = colon + 1;
}
}
node = n;
} else {
if (debug)
fprintf(stderr, "target for %s is default (%s)\n", envname, defaultnode);
node = defaultnode;
}
if (service == NULL) {
service = defaultservice;
}
return getaddrinfo(node, service, &hints, res);
}
/* *************** streamstate *************** */
void stream_dump(FILE *f, struct streamstate *s) {
fprintf(f, "[%d](%p) in: %p out %p\n", s->id, (void*)s, (void*)&(s->x[0].stream), (void*)&(s->x[1].stream));
}
struct streamstate *new_streamstate(struct demuxer *demux) {
int err;
struct streamstate *ret = calloc(1, sizeof(struct streamstate));
if (ret) {
if ((err = uv_timer_init(demux->loop, &(ret->timer)))) {
fprintf(stderr, "[%d] Failed to uv_timer_init: (%d) %s\n", ret->id,
err, uv_strerror(err));
free(ret);
return NULL;
}
ret->timer.data = ret;
ret->surmised_protocol = DEMUX_UNKNOWN;
ret->id = demux->next_stream_id++;
demux->active_streams++;
for (int ix = 0; ix < 2; ix++) {
if ((err = uv_tcp_init(demux->loop, &(ret->x[ix].stream)))) {
fprintf(stderr, "[%d] Failed to uv_tcp_init %s: (%d) %s\n", ret->id,
(ix? "outbound":"inbound"), err, uv_strerror(err));
free(ret);
return NULL;
}
ret->x[ix].stream.data = ret;
}
touch_streamstate(ret);
}
return ret;
}
void free_streamstate(struct streamstate *st) {
if (!st->released) {
st->released = true;
for (int ix = 0; ix < 2; ix++)
uv_close((uv_handle_t*) (&(st->x[ix].stream)), close_streamstate_stream);
uv_close((uv_handle_t*)(&(st->timer)), close_streamstate_timer);
}
}
void touch_streamstate(struct streamstate *st) {
int err;
struct demuxer *demux = st->x[0].stream.loop->data;
st->last_touched = uv_now(demux->loop);
if ((err = uv_timer_stop(&(st->timer))))
fprintf(stderr, "[%d] failed to uv_timer_stop: (%d) %s\n", st->id, err, uv_strerror(err));
if (demux->stream_timeout > 0)
if ((err = uv_timer_start(&(st->timer), streamstate_timeout, demux->stream_timeout, 0)))
fprintf(stderr, "[%d] failed to uv_timer_start(%llu): (%d) %s\n",
st->id, (long long unsigned int)(demux->stream_timeout),
err, uv_strerror(err));
}
/* *************** demuxer *************** */
struct demuxer* new_demuxer(uv_loop_t *loop, int max_listeners) {
int err;
struct demuxer *ret = calloc(1, sizeof(struct demuxer));
if (ret == NULL)
return NULL;
ret->listeners = calloc(max_listeners, sizeof(uv_tcp_t));
if (ret->listeners == NULL) {
free(ret);
return NULL;
}
ret->max_listeners = max_listeners;
ret->loop = loop;
struct { uv_signal_t* sig; int signum; uv_signal_cb handler; } sigs[] = {
{ &(ret->usr1_signal), SIGUSR1, demuxer_signal_usr1 },
{ &(ret->term_signal), SIGTERM, demuxer_signal_term }
};
for (int ix = 0; ix < sizeof(sigs)/sizeof(sigs[0]); ix++) {
if ((err = uv_signal_init(loop, sigs[ix].sig))) {
fprintf(stderr, "Failed to init signal handler: (%d) %s\n", err, uv_strerror(err));
free(ret);
return NULL;
}
if ((err = uv_signal_start(sigs[ix].sig, sigs[ix].handler, sigs[ix].signum))) {
fprintf(stderr, "Failed to connect signal handler (%d): (%d) %s\n", sigs[ix].signum, err, uv_strerror(err));
free(ret);
return NULL;
}
}
loop->data = ret;
return ret;
}
void demuxer_going_down(uv_handle_t *h, void *x) {
if (h == NULL || x == NULL || h->data == NULL)
return;
struct demuxer *demux = x;
if (h->loop != demux->loop)
return;
if (h->type != UV_TCP)
return;
struct streamstate *st = h->data;
free_streamstate(st);
}
void free_demuxer(struct demuxer* demux) {
uv_walk(demux->loop, demuxer_going_down, demux);
for (int i = 0; i < demux->listener_count; i++) {
uv_close((uv_handle_t*)(demux->listeners + i), NULL);
}
uv_close((uv_handle_t*)(&demux->usr1_signal), NULL);
uv_close((uv_handle_t*)(&demux->term_signal), NULL);
int err = uv_run(demux->loop, UV_RUN_NOWAIT);
if (demux->debug)
fprintf(stderr, "final loop instance completed: %d (%s)\n", err,
err ? (err < 0 ? uv_strerror(err) : "unknown") : "none");
free(demux->listeners);
free(demux);
}
static void demuxer_rotate(struct demuxer *demuxer, enum protocol proto) {
if (proto == DEMUX_HTTP) {
demuxer->next_http = demuxer->next_http->ai_next;
if (demuxer->next_http == NULL)
demuxer->next_http = demuxer->http_targets;
} else if (proto == DEMUX_DNS) {
demuxer->next_dns = demuxer->next_dns->ai_next;
if (demuxer->next_dns == NULL)
demuxer->next_dns = demuxer->dns_targets;
} else {
assert(false);
}
}
static const struct addrinfo* demuxer_getaddrinfo(const struct demuxer *demuxer, enum protocol proto) {
if (proto == DEMUX_HTTP)
return demuxer->next_http;
else if (proto == DEMUX_DNS)
return demuxer->next_dns;
else
assert(false);
}
void demuxer_signal_usr1(uv_signal_t *sig, int signum) {
assert(sig != NULL);
assert(signum = SIGUSR1);
struct demuxer *demux = sig->loop->data;
demuxer_dump(demux, stderr);
}
void demuxer_signal_term(uv_signal_t *sig, int signum) {
assert(sig != NULL);
assert(signum = SIGTERM);
uv_stop(sig->loop);
}
static inline void showtime(FILE *f, const char *label, int id, uint64_t t, uint64_t now) {
assert(now >= t);
fprintf(f, "%s: [%d] last_touched: %g seconds ago\n",
label, id, (double)(now-t)/1000);
}
static void demuxer_dump(struct demuxer *demuxer, FILE *f) {
const struct addrinfo *x;
uint64_t now;
now = uv_now(demuxer->loop);
struct min_max_streamstate s = { .min = NULL, .max = NULL };
uv_walk(demuxer->loop, get_min_max_streamstate, &s);
fprintf(f, "Total created: %d\nOutstanding: %d\nDefault timeout: %llums\n",
demuxer->next_stream_id, demuxer->active_streams, (long long unsigned int)demuxer->stream_timeout);
if (s.min)
showtime(f, "oldest", s.min->id, s.min->last_touched, now);
if (s.max)
showtime(f, "youngest", s.max->id, s.max->last_touched, now);
for (x = demuxer->dns_targets; x; x = x->ai_next)
dump_addrinfo(f, "DNS", x, x == demuxer->next_dns);
for (x = demuxer->http_targets; x; x = x->ai_next)
dump_addrinfo(f, "HTTP", x, x == demuxer->next_http);
}
int demuxer_add_listener_fd(struct demuxer *demuxer, int fd, int listen_backlog) {
int err;
if (sd_is_socket(fd, AF_UNSPEC, SOCK_STREAM, 1) <= 0) {
fprintf(stderr, "File descriptor %d is not a stream socket, ignoring.\n", fd);
return 0;
}
if (demuxer->listener_count >= demuxer->max_listeners) {
fprintf(stderr, "too many listeners! (have: %d, max: %d) ignoring file descriptor %d\n",
demuxer->listener_count, demuxer->max_listeners, fd);
return 0;
}
if ((err = uv_tcp_init(demuxer->loop, demuxer->listeners + demuxer->listener_count))) {
fprintf(stderr, "failed to uv_tcp_init: %s\n", uv_strerror(err));
return 1;
}
/* listeners have NULL data, compared with normal streams, which
point to their enclosing struct streamstate */
demuxer->listeners[demuxer->listener_count].data = NULL;
if ((err = uv_tcp_open(demuxer->listeners + demuxer->listener_count, fd))) {
fprintf(stderr, "Failed to uv_tcp_open (fd: %d): %s\n", fd, uv_strerror(err));
return 1;
}
if ((err = uv_listen((uv_stream_t*)(demuxer->listeners + demuxer->listener_count),
listen_backlog, new_inbound))) {
fprintf(stderr, "Failed to listen on stream (fd: %d): %s\n", fd, uv_strerror(err));
return 1;
}
demuxer->listener_count++;
return 0;
}
/* *************** workflow stages *************** */
void new_inbound(uv_stream_t* server, int status) {
assert(server != NULL);
if (status) {
fprintf(stderr, "ignoring weird non-zero new_inbound() status: (%d) %s\n", status, uv_strerror(status));
} else {
struct streamstate *st = new_streamstate(server->loop->data);
int err;
if (st == NULL) {
fprintf(stderr, "No RAM for new connection\n");
return;
}
if ((err = uv_accept(server, (uv_stream_t*)(&(st->x[0].stream))))) {
fprintf(stderr, "failed to uv_accept: %s\n", uv_strerror(err));
free_streamstate(st);
return;
}
st->x[0].initialized = true;
if ((err = uv_read_start((uv_stream_t*)(&(st->x[0].stream)), first_alloc, inbound_first_read))) {
fprintf(stderr, "failed to uv_read_start: %s\n", uv_strerror(err));
free_streamstate(st);
return;
}
}
}
void first_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf) {
struct streamstate *st = handle->data;
assert(st != NULL);
assert((uv_handle_t*)(&st->x[0].stream) == handle);
if (st->x[0].staging == NULL) {
if (suggested_size < INTERESTING_OCTETS)
suggested_size = INTERESTING_OCTETS;
st->x[0].staging = malloc(suggested_size);
if (st->x[0].staging == NULL)
fprintf(stderr, "first_alloc malloc() failed: (%d) %s\n", errno, strerror(errno));
st->first_read_size = suggested_size;
}
buf->base = (char*)(st->x[0].staging) + st->x[0].bytes_read;
if (buf->base)
buf->len = st->first_read_size - st->x[0].bytes_read;
else
buf->len = 0;
}
void inbound_first_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf) {
int err;
assert(stream != NULL);
assert(stream->data != NULL);
struct streamstate *st = stream->data;
assert((uv_stream_t*)(&(st->x[0].stream)) == stream); /* this must only be called on input! */
assert(st->x[0].initialized);
assert(st->surmised_protocol == DEMUX_UNKNOWN);
assert((unsigned char*)(buf->base) == st->x[0].staging + st->x[0].bytes_read);
struct demuxer *demuxer = stream->loop->data;
if (nread == 0 || nread == UV_EAGAIN || nread == UV_EBUSY) {
fprintf(stderr, "[%d] ignoring weird inbound_first_read() with status: (%zd), %s\n",
st->id, nread, uv_strerror(nread));
return;
}
if (nread < 0) {
fprintf(stderr, "[%d] error on inbound stream before we could interpret it (%zd): %s\n", st->id, nread, uv_strerror(nread));
free_streamstate(st);
return;
}
if (demuxer->debug)
fprintf(stderr, "[%d] inbound initial read %zd octets\n", st->id, nread);
st->x[0].bytes_read += nread;
st->surmised_protocol = surmise_protocol(st->x[0].staging, st->x[0].bytes_read);
if (st->surmised_protocol == DEMUX_UNKNOWN) {
fprintf(stderr, "[%d] inbound stream is still unknown: only sent %zd octets so far (%zd in this flight)\n",
st->id, st->x[0].bytes_read, nread);
touch_streamstate(st);
return;
}
/* do not read more from the inbound stream until we've established
the outbound stream */
if ((err = uv_read_stop(stream))) {
fprintf(stderr, "[%d] inbound initial uv_read_stop error %d: %s\n", st->id, err, uv_strerror(err));
free_streamstate(st);
return;
}
const struct addrinfo *addr = demuxer_getaddrinfo(demuxer, st->surmised_protocol);
if (demuxer->debug) {
fprintf(stderr, "[%d] new stream is %s\n", st->id, protocol_names[st->surmised_protocol]);
dump_addrinfo(stderr, protocol_names[st->surmised_protocol], addr, false);
}
demuxer_rotate(demuxer, st->surmised_protocol);
if ((err = uv_tcp_connect(&st->connect, &(st->x[1].stream),
addr->ai_addr, outbound_connection))) {
fprintf(stderr, "[%d] Failed to uv_tcp_connect during new outbound stream: (%d) %s\n",
st->id, err, uv_strerror(err));
free_streamstate(st);
}
touch_streamstate(st);
}
void outbound_connection(uv_connect_t *req, int status) {
int err;
assert(req != NULL);
assert(req->handle != NULL);
struct streamstate *st = req->handle->data;
assert(st != NULL);
assert(((uv_stream_t*)(&(st->x[1]))) == req->handle); /* this should be the outbound leg */
assert(!st->x[1].initialized);
if (status) {
fprintf(stderr, "[%d] failed to complete the connection: status %d (%s)\n",
st->id, status, uv_strerror(status));
free_streamstate(st);
return;
}
st->x[1].initialized = true;
/* send the interesting bits along */
const struct uv_buf_t buf = { .base = (char*)(st->x[0].staging), .len = st->x[0].bytes_read };
if ((err = uv_write(&(st->x[1].writer), (uv_stream_t*)(&(st->x[1].stream)),
&buf, 1, write_complete))) {
fprintf(stderr, "[%d] Failed to uv_write first frame to outbound: (%d) %s\n",
st->id, err, uv_strerror(err));
free_streamstate(st);
return;
}
/* start reading from outbound: */
if ((err = uv_read_start((uv_stream_t*)(&(st->x[1].stream)), stream_alloc, stream_read))) {
fprintf(stderr, "[%d] Failed to uv_read_start outbound: (%d) %s\n",
st->id, err, uv_strerror(err));
free_streamstate(st);
return;
}
touch_streamstate(st);
}
void stream_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf) {
struct streamstate *st = handle->data;
assert(st != NULL);
for (int ix = 0; ix < 2; ix++)
if ((uv_tcp_t*)handle == &(st->x[ix].stream)) {
assert(st->x[ix].staging == NULL);
st->x[ix].staging = malloc(suggested_size);
if (st->x[ix].staging == NULL)
fprintf(stderr, "stream_alloc malloc() failed: (%d) %s\n", errno, strerror(errno));
buf->base = (char*)(st->x[ix].staging);
buf->len = buf->base ? suggested_size : 0;
}
}
void write_complete(uv_write_t* write, int status) {
int err;
assert(write != NULL);
assert(write->handle != NULL);
struct streamstate *st = write->handle->data;
assert(st != NULL);
assert(st->x[0].initialized);
assert(st->x[1].initialized);
for (int ix = 0; ix < 2; ix++)
if ((uv_tcp_t*)(write->handle) == &(st->x[ix].stream)) {
/* free the buffer from the other since it has been written: */
assert(st->x[!ix].staging != NULL);
free(st->x[!ix].staging);
st->x[!ix].staging = NULL;
if (status) {
fprintf(stderr, "[%d] write to %s failed: (%d) %s\n", st->id, (ix?"outbound":"inbound"),
status, uv_strerror(status));
free_streamstate(st);
return;
}
/* re-enable reading from the other */
if ((err = uv_read_start((uv_stream_t*)&(st->x[!ix].stream), stream_alloc, stream_read))) {
fprintf(stderr, "[%d] re-enabling reading from %s failed: (%d) %s\n", st->id, (!ix?"outbound":"inbound"),
status, uv_strerror(status));
free_streamstate(st);
return;
}
touch_streamstate(st);
return;
}
assert(false); /* should never reach here */
}
void stream_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf) {
assert(stream != NULL);
struct streamstate *st = stream->data;
assert(st != NULL);
assert(st->x[0].initialized);
assert(st->x[1].initialized);
struct demuxer *demuxer = stream->loop->data;
assert(demuxer != NULL);
int err;
for (int ix = 0; ix < 2; ix++)
if ((uv_tcp_t*)stream == &(st->x[ix].stream)) {
const char *indir = (ix? "outbound" : "inbound");
const char *outdir = (ix? "inbound" : "outbound");
assert(!st->x[ix].eof_seen);
touch_streamstate(st);
if (nread == 0 || nread == UV_EAGAIN || nread == UV_EBUSY ||
nread ==UV_ENOBUFS || nread == UV_EINTR) {
fprintf(stderr, "[%d] ignoring weird stream_read() with status: (%zd), %s\n",
st->id, nread, uv_strerror(nread));
return;
}
if (nread == UV_EOF) {
if (demuxer->debug)
fprintf(stderr, "[%d] EOF received from %s leg\n", st->id, indir);
/* we've just seen the EOF from one channel -- shut down the
write side of the other. */
st->x[ix].eof_seen = true;
if ((err = uv_shutdown(&(st->x[!ix].shutdown), (uv_stream_t*)(&(st->x[!ix].stream)), after_shutdown))) {
fprintf(stderr, "[%d] error calling uv_shutdown() on %s leg: (%d) %s\n", st->id, outdir, err, uv_strerror(err));
free_streamstate(st);
}
return;
}
if (nread < 0) {
fprintf(stderr, "[%d] closing due to %s read error: (%zd): %s\n",
st->id, indir, nread, uv_strerror(nread));
free_streamstate(st);
return;
}
if (demuxer->debug)
fprintf(stderr, "[%d] %s read %zd octets\n", st->id, indir, nread);
assert((unsigned char*)(buf->base) == st->x[ix].staging);
if ((err = uv_read_stop(stream))) {
fprintf(stderr, "[%d] %s uv_read_stop error %d: %s\n", st->id, indir, err, uv_strerror(err));
free_streamstate(st);
return;
}
st->x[ix].bytes_read += nread;
const struct uv_buf_t buf = { .base = (char*)(st->x[ix].staging), .len = nread };
/* schedule write to other channel */
if ((err = uv_write(&(st->x[!ix].writer), (uv_stream_t*)&(st->x[!ix].stream),
&buf, 1, write_complete))) {
fprintf(stderr, "[%d] %s uv_write failed: (%d) %s\n", st->id, outdir, err, uv_strerror(err));
free_streamstate(st);
return;
}
return;
}
assert(false); /* should never reach here */
}
void after_shutdown(uv_shutdown_t* req, int status) {
assert(req != NULL);
assert(req->handle != NULL);
assert(req->handle->data != NULL);
struct streamstate *st = req->handle->data;
assert(st->x[0].initialized);
assert(st->x[1].initialized);
for (int ix = 0; ix < 2; ix++)
if ((uv_tcp_t*)(req->handle) == &(st->x[ix].stream)) {
if (status) {
fprintf(stderr, "[%d] error in shutdown of %s stream: (%d) %s\n", st->id,
(ix?"outbound":"inbound"), status, uv_strerror(status));
free_streamstate(st);
return;
}
/* if we've seen eof on both sides, we can shut this down */
if (st->x[!ix].eof_seen)
free_streamstate(st);
else
touch_streamstate(st);
return;
}
assert(false); /* should never have gotten here. */
}
inline void streamstate_dispose(struct streamstate *st, struct demuxer *demux) {
if (st->timer_closed &&
st->x[0].closed &&
st->x[1].closed) {
demux->active_streams--;
for (int ix = 0; ix < 2; ix++)
if (st->x[ix].staging) {
free(st->x[ix].staging);
st->x[ix].staging = NULL;
}
free(st);
}
}
void close_streamstate_timer(uv_handle_t *timer) {
assert(timer != NULL);
assert(timer->data != NULL);
struct streamstate *st = timer->data;
assert(timer->loop != NULL);
struct demuxer *demux = timer->loop->data;
if (demux->debug)
fprintf(stderr, "[%d] close_streamstate_timer\n", st->id);
st->timer_closed = true;
streamstate_dispose(st, demux);
}
void close_streamstate_stream(uv_handle_t *stream) {
assert(stream != NULL);
assert(stream->data != NULL);
struct streamstate *st = stream->data;
int found = -1;
assert(stream->loop != NULL);
struct demuxer *demux = stream->loop->data;
assert(demux != NULL);
for (int ix = 0; ix < 2; ix++)
if ((uv_tcp_t*)stream == &(st->x[ix].stream)) {
found = ix;
}
if (demux->debug)
fprintf(stderr, "[%d] close_streamstate_stream %d\n", st->id, found);
assert(found != -1);
st->x[found].closed = true;
streamstate_dispose(st, demux);
}
void streamstate_timeout(uv_timer_t *timer) {
assert(timer != NULL);
assert(timer->data != NULL);
struct streamstate *st = timer->data;
assert(timer->loop != NULL);
struct demuxer *demux = timer->loop->data;
assert(demux != NULL);
if (demux->debug)
fprintf(stderr, "[%d] closing stream due to IDLE_TIMEOUT\n", st->id);
free_streamstate(st);
}
void get_min_max_streamstate(uv_handle_t *h, void *x) {
if (h->type != UV_TCP)
return;
if (h->data == NULL)
return;
if (h->data == h->loop)
return;
if (x == NULL)
return;
struct min_max_streamstate *minmax = x;
struct streamstate *s = h->data;
if (minmax->min == NULL || minmax->min->last_touched > s->last_touched)
minmax->min = s;
if (minmax->max == NULL || minmax->max->last_touched < s->last_touched)
minmax->max = s;
}
int main(int argc, const char **argv) {
int lmax = 0;
int err;
uv_loop_t loop;
int listen_backlog = 10;
struct demuxer *demuxer = NULL;
if ((err = uv_loop_init(&loop)) < 0) {
fprintf(stderr, "Failed to initialize uv_loop: %s\n", uv_strerror(err));
return 1;
}
lmax = sd_listen_fds(0);
demuxer = new_demuxer(&loop, lmax);
if ((err = setvbuf(stderr, NULL, _IONBF, 0))) {
fprintf(stderr, "Failed to unbuffer stderr: %s\n", strerror(errno));
return 1;
}
for (int i = 0; i < lmax; i++) {
int fd = SD_LISTEN_FDS_START + i;
if ((err = demuxer_add_listener_fd(demuxer, fd, listen_backlog))) {
fprintf(stderr, "Failed to listen on fd %d: (%d) %s\n", fd, err, uv_strerror(err));
return 1;
}
}
if (demuxer->listener_count < 1) {
fprintf(stderr, "No listening sockets available! nothing to do.\n");
return 1;
}
char *debug = getenv("HDDEMUX_DEBUG");
demuxer->debug = (debug != NULL && debug[0] != '\0');
char *timeout = getenv("IDLE_TIMEOUT");
if (timeout)
demuxer->stream_timeout = strtoull(timeout, NULL, 0);
else
demuxer->stream_timeout = DEFAULT_IDLE_TIMEOUT;
if ((err = get_target_addrinfo("HTTP_TARGET", "localhost", "80", &demuxer->http_targets, demuxer->debug))) {
fprintf(stderr, "Failed looking up HTTP_TARGET: %s\n", gai_strerror(err));
return 1;
}
if ((err = get_target_addrinfo("DNS_TARGET", "localhost", "53", &demuxer->dns_targets, demuxer->debug))) {
fprintf(stderr, "Failed looking up DNS_TARGET: %s\n", gai_strerror(err));
return 1;
}
demuxer->next_http = demuxer->http_targets;
demuxer->next_dns = demuxer->dns_targets;
if (demuxer->debug)
demuxer_dump(demuxer, stderr);
sd_notify(0, "READY=1");
err = uv_run(&loop, UV_RUN_DEFAULT);
if (demuxer->debug)
fprintf(stderr, "loop completed: %d (%s)\n", err, err ? (err < 0 ? uv_strerror(err) : "unknown") : "none");
freeaddrinfo(demuxer->http_targets);
freeaddrinfo(demuxer->dns_targets);
free_demuxer(demuxer);
if ((err = uv_loop_close(&loop)))
fprintf(stderr, "uv_loop_close() failed: (%d) %s\n", err, uv_strerror(err));
return 0;
}
hddemux-0.5/hddemux.conf 0000664 0000000 0000000 00000000175 14176035274 0015335 0 ustar 00root root 0000000 0000000 # tmpfiles.d(5) runtime directory for hddemux
d /run/hddemux 0751 root root - -
d /run/hddemux/workdir 0750 root hddemux - -
hddemux-0.5/hddemux.service 0000664 0000000 0000000 00000000373 14176035274 0016050 0 ustar 00root root 0000000 0000000 [Unit]
Description=HTTP/1.x and DNS demuxer service
Documentation=man:hddemux(1)
Requires=hddemux.socket
[Service]
Type=notify
ExecStart=/usr/bin/hddemux
WorkingDirectory=/run/hddemux/workdir
User=hddemux
Group=hddemux
[Install]
Also=hddemux.socket
hddemux-0.5/hddemux.socket 0000664 0000000 0000000 00000000236 14176035274 0015676 0 ustar 00root root 0000000 0000000 [Unit]
Description=HTTP/1.x and DNS demuxer socket
Documentation=man:hddemux(1)
[Socket]
ListenStream=/run/hddemux/socket
[Install]
WantedBy=sockets.target
hddemux-0.5/testsuite 0000775 0000000 0000000 00000010676 14176035274 0015016 0 ustar 00root root 0000000 0000000 #!/bin/bash
# test suite for hddemux
# requires:
# - nginx
# - knot-resolver
# - kdig (from knot-dnsutils)
# - curl
# - certtool (from gnutls-bin)
# environment variables:
# - WORKDIR: a place for all generated files.
# if unset, it will be auto-generated.
# it will be created as needed.
# if the directory doesn't currently exist, it will be cleaned up at exit.
# if it already exists, it will not be cleaned up.
# - TESTIP: the IP address to use for testing.
# the user needs to be able to open listening sockets, and to connect to them
# by default, choose a random IP in 127.0.0.0/8
# Author: Daniel Kahn Gillmor
# 2018-08-29
# License: GPLv3+
# error on exit
set -e
# for handling jobspecs:
set -m
# Unset proxy to make sure curl behaves correctly
unset https_proxy http_proxy
hddemux=$(which hddemux) || hddemux=./hddemux
[ -x "$hddemux" ]
if [ -z "$WORKDIR" ]; then
d="$(mktemp -d)"
remove="$d"
else
d="$WORKDIR"
fi
ip="${TESTIP:-127.$(( $RANDOM % 256 )).$(( $RANDOM % 256 )).$(( $RANDOM % 256 ))}"
printf "hddemux test\n------------\n binary: %s\n workdir: %s\n IP addr: %s\n" "$hddemux" "$d" "$ip"
section() {
printf "\n%s\n" "$1"
sed 's/./-/g' <<<"$1"
}
cleanup () {
section "cleaning up"
find "$d" -ls
tail "$d/"*.err
/usr/sbin/nginx -c "$d/nginx.conf" -p "$d" -s stop 2> "$d/nginx-stop.err" || true
kill %2 || true
kill %1 || true
if [ "$remove" ]; then
printf "cleaning up working directory %s\n" "$remove"
rm -rf "$remove"
fi
}
trap cleanup EXIT
section "simple failing run"
# hddemux with no arguments and no listening file descriptors should fail:
if "$hddemux" 2>&1; then
false
fi
section "make Certificate Authority key and certificate"
cat > "$d/ca.template" < "$d/ee.template" < "$d/kresd.conf" < iterate' }
net.tls("$d/ee-cert.pem", "$d/ee-key.pem")
hints["monkeys.example"] = "127.15.23.5"
EOF
/usr/sbin/kresd --config "$d/kresd.conf" --tls "$ip@8853" --noninteractive "$d" 2> "$d/kresd.err" &
section "make hddemux configuration on $ip:2000"
systemd-socket-activate -l "$ip:2000" -E HTTP_TARGET="$ip:8853" -E DNS_TARGET="$ip:8853" "$hddemux" 2> "$d/hddemux.err" &
section "set up nginx on $ip:4433"
mkdir -p "$d/nginx"
cat >"$d/nginx.conf" < "$d/data/index.txt"
/usr/sbin/nginx -c "$d/nginx.conf" -p "$d" 2> "$d/nginx.err"
section "test with kdig"
x=$(kdig +short +tls +tls-ca="$d/ca-cert.pem" +tls-hostname=test.example @"$ip:2000" monkeys.example)
[ "$x" = "127.15.23.5" ]
echo "successful DNS-over-TLS request to $ip on port 2000"
section "test with curl"
x=$(curl --silent --show-error --cacert "$d/ca-cert.pem" --resolve "test.example:2000:$ip" --resolve "test.example:4433:$ip" https://test.example:4433/)
[ "$x" = "Hello, world!" ]
echo "successful HTTPS request to $ip on port 2000"