pax_global_header00006660000000000000000000000064141760352740014523gustar00rootroot0000000000000052 comment=073d155a46fdf303ab8462f6e49665666e7f169f hddemux-0.5/000077500000000000000000000000001417603527400130255ustar00rootroot00000000000000hddemux-0.5/.gitignore000066400000000000000000000002151417603527400150130ustar00rootroot00000000000000*~ hddemux hddemux.1 .refcache/ draft-dkg-dprive-demux-dns-http.xml draft-dkg-dprive-demux-dns-http.html draft-dkg-dprive-demux-dns-http.txt hddemux-0.5/LICENSE000066400000000000000000001045171417603527400140420ustar00rootroot00000000000000 GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . hddemux-0.5/Makefile000066400000000000000000000017551417603527400144750ustar00rootroot00000000000000#!/usr/bin/make -f CFLAGS += -D_GNU_SOURCE -g -O3 PKG_CONFIG ?= pkg-config CFLAGS += $(shell $(PKG_CONFIG) --cflags libuv) CFLAGS += $(shell $(PKG_CONFIG) --cflags libsystemd) LDFLAGS += $(shell $(PKG_CONFIG) --libs libuv) LDFLAGS += $(shell $(PKG_CONFIG) --libs libsystemd) all: hddemux hddemux.1 check: hddemux PATH=.:$$PATH ./testsuite hddemux: hddemux.c $(CC) $(CPPFLAGS) $(CFLAGS) $< -Wl,--as-needed $(LDFLAGS) -std=c11 -pedantic -Wall -Werror -o $@ hddemux.1: hddemux.1.md pandoc -s -f markdown -t man -o $@ $< draft-dkg-dprive-demux-dns-http.xml: draft-dkg-dprive-demux-dns-http.md kramdown-rfc2629 < $< > $@ draft-dkg-dprive-demux-dns-http.html: draft-dkg-dprive-demux-dns-http.xml xml2rfc $< --html draft-dkg-dprive-demux-dns-http.txt: draft-dkg-dprive-demux-dns-http.xml xml2rfc $< --text clean: rm -rf hddemux hddemux.1 \ .refcache/ \ draft-dkg-dprive-demux-dns-http.txt \ draft-dkg-dprive-demux-dns-http.html \ draft-dkg-dprive-demux-dns-http.xml .PHONY: clean all check hddemux-0.5/README.md000066400000000000000000000016011417603527400143020ustar00rootroot00000000000000Given restrictive firewalls and intrusive network monitors, it can be both privacy-preserving and connectivity-enhancing to be able to serve both HTTPS (HTTP/1.x) and DNS-over-TLS from the same TCP port. This project aims to document specifically how that can be done safely, and to provide simple code to demonstrate the mechanism. As of this writing, the code is deployed and in-use at the host `dns.cmrg.net`, which offers both HTTPS and a full recursive resolver (via DNS-over-TLS), both on port 443. Please see https://dns.cmrg.net for more details of that service. This project includes: * `hddemux.c` -- a `libuv`-based HTTP/1.x and DNS multiplexing server * `hddemux.socket` and `hddemux.service` -- systemd units to manage `hddemux` * `draft-dkg-dprive-demux-dns-http.md` -- an Internet Draft documenting the rationale and the algorithm. Patches and commentary welcome! hddemux-0.5/TODO.md000066400000000000000000000012111417603527400141070ustar00rootroot00000000000000* Cleanup outstanding idle connections more cleverly -- can we only do this cleanup when under memory pressure? is there a better way to choose which connections to kill? IDLE_TIMEOUT is a pretty clumsy knob compared to ideal memory management practices. * Enable unix-domain sockets in the HTTP_TARGET and DNS_TARGET backends -- no reason that these need to be TCP addresses. * Are there other client-speaks-first streaming protocols that we can demux in addition to HTTP and DNS? * Consider making hddemux work on non-systemd systems (e.g. pass a set of listener arguments on the CLI in addition to looking for SD_LISTEN_FDS) hddemux-0.5/draft-dkg-dprive-demux-dns-http.md000066400000000000000000000702061417603527400213650ustar00rootroot00000000000000--- title: Demultiplexing Streamed DNS from HTTP/1.x docname: draft-dkg-dprive-demux-dns-http-03 date: 2017-05-17 category: info ipr: trust200902 area: int workgroup: dprive keyword: Internet-Draft updates: 1035, 7230 stand_alone: yes pi: [toc, sortrefs, symrefs] author: - ins: D. K. Gillmor name: Daniel Kahn Gillmor org: American Civil Liberties Union street: 125 Broad St. city: New York, NY code: 10004 country: USA abbrev: ACLU email: dkg@fifthhorseman.net informative: RFC3007: RFC5246: RFC6840: RFC6895: RFC7301: RFC7540: RFC7830: RFC7858: I-D.ietf-dnsop-dns-wireformat-http: I-D.hoffman-dns-over-https: HAPROXY: target: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt title: The Proxy protocol author: name: Willy Tarreau ins: W. Tarreau org: HAProxy Technologies date: 2017-03-10 normative: RFC1035: RFC1945: RFC2119: RFC2136: RFC5234: RFC7230: --- abstract DNS over TCP and HTTP/1.x are both stream-oriented, client-speaks-first protocols. They can both be run over a stream-based security protocol like TLS. A server accepting a stream-based client can distinguish between a valid stream of DNS queries and valid stream of HTTP/1.x requests by simple observation of the first few octets sent by the client. This can be done without any external demultiplexing mechanism like TCP port number or ALPN. Implicit multiplexing of the two protocols over a single listening port can be useful for obscuring the presence of DNS queries from a network observer, which makes it relevant for DNS privacy. Widespread adoption of the described approach could constrain evolution of the stream-based variants of both DNS (RFC1035) and HTTP/1.x (RFC7230) by ossifying existing distinct bit patterns found in early octets sent by the client. However, this draft explicitly rules out multiplexing in this form with HTTP/2, so it should place no constraints on the evolution of HTTP/2 or any higher version of HTTP. --- middle Introduction ============ DNS and HTTP/1.x are both client-speaks-first protocols capable of running over stream-based transport like TCP, or as the payload of a typical TLS {{RFC5246}} session. There are some contexts where it is useful for a server to be able to decide what protocol is used by an incoming TCP stream, to choose dynamically between DNS and HTTP/1.x on the basis of the stream itself (rather than a port designation or other explicit demultiplexing). For example, a TLS terminator listening on port 443 and receiving either no ALPN token at all, or the "http/1.1" ALPN token might be willing to serve DNS-over-TLS {{RFC7858}} as well as HTTPS. A simple demultiplexing server should do this demuxing based on the first few bytes sent by the client on a given stream; once a choice has been established, the rest of the stream is committed to one or the other interpretation. This document provides proof that a demultiplexer can robustly distinguish HTTP/1.x from DNS on the basis of the content of the first few bytes of the client's stream alone. A DNS client that knows it is talking to a server which is this position (e.g. trying to do DNS-over-TLS on TCP port 443 with no ALPN token, used traditionally only for HTTPS) might also want to be aware of network traffic patterns that could confuse such a server. This document presents explicit mitigations that such a DNS client MAY decide to use. This document limits its discussion to HTTP/1.x over TCP or TLS or some other classical stream-based protocol (it excludes HTTP over QUIC, for example, and HTTP/2 {{RFC7540}} or later). Likewise, it considers only the TCP variant of DNS (and excludes DNS over UDP or any other datagram transport). Terminology ----------- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in {{RFC2119}}. Scoping ======= Distinguish only at the start of a stream ----------------------------------------- A server which attempts to distinguish DNS queries from HTTP/1.x requests individually might consider using these guidelines in the middle of a running stream (e.g. at natural boundaries, like the end of an HTTP/1.1 request, or after a DNS message), but this document focuses specifically on a heuristic choice for the whole stream, based on the initial few octets sent by the client. While it's tempting to consider distinguishing at multiple points in the stream, the complexities of determining the specific end of an HTTP/1.x request body and handling HTTP/1.x error cases make this more difficult to implement on the side of a DNS client configured to talk to such a server. Interleaving the responses themselves on a stream with multiple data elements is also challenging. So do not use this technique anywhere but at the beginning of a stream! If being able to interleave DNS queries with HTTP requests on a single stream is desired, a strategy like {{I-D.hoffman-dns-over-https}} or {{I-D.ietf-dnsop-dns-wireformat-http}} is recommended instead. HTTP/2 is not always client-speaks-first ---------------------------------------- While this demultiplexing technique functions for HTTP/1.0 and HTTP/1.1, it does not work for HTTP/2 {{RFC7540}} because HTTP/2 is not guaranteed to be a client-speaks-first protocol. In particular, many HTTP/2 servers prefer to send a SETTINGS frame immediately without waiting for data from the client, if they already know they're speaking HTTP/2. In the event that HTTP/2 is to be transported over TLS, the ALPN token negotiated in the TLS handshake is "h2", which allows the server to know as soon as the handshake is complete that it can start pushing data to the client. A standard DNS-over-TLS client connecting to a server that might be multiplexing DNS with HTTP on the same listener MUST NOT indicate an intent to speak HTTP/2 that could prompt this unsolicited first flight from the server. Concretely, a DNS client connecting over TLS on TCP port 443 expecting to speak standard DNS-over-TLS {{RFC7858}} MUST NOT offer or accept the "h2" ALPN token. If use of DNS in the same channel as HTTP/2 is deisred, a strategy like {{I-D.hoffman-dns-over-https}} is recommended instead. Avoid multiplexing in the clear ------------------------------- The widespread deployment of transparent HTTP/1.x proxies makes it likely that any attempt to do this kind of multiplexing/demultiplexing on a cleartext channel that normally carries HTTP/1.x (e.g. TCP port 80) will fail or trigger other "interesting" behaviors. The approach described in this draft should be done only in channels sufficiently obscured that a transparent proxy would not try to interpret the resultant stream. Avoid mixing with other demultiplexing -------------------------------------- Some other (non-IETF) systems (e.g. {{HAPROXY}}) take a similar approach with multiplexing data on top of HTTP/1.x by taking advantage of bitpatterns that are presumed to not be present in normal HTTP/1.x requests. Use of the approach described in this draft in conjunction with these other approaches is not advisable. Doing so safely would require explicit and detailed review of all three (or more) protocols involved. Heavily-restricted network environments --------------------------------------- Some network environments are so tightly constrained that outbound connections on standard TCP ports are not accessible. In some of these environments, an explicit HTTP proxy is available, and clients must use the HTTP CONNECT pseudo-method to make https connections. While this multiplexing approach can be used in such a restrictive environment, it would be necessary to teach the DNS client how to talk to (and through) the HTTP proxy. These details are out of scope for this document. A DNS client capable of this additional layer of complexity may prefer to pursue a strategy like {{I-D.hoffman-dns-over-https}} instead. Why not ALPN? ------------- If this is done over TLS, a natural question is whether the client should simply indicate its preferred protocol in the TLS handshake's ALPN {{RFC7301}} extension (e.g. with some new ALPN token "dns"). However, ALPN tokens requested by the client are visible to a network observer (and the ALPN token selected by the server is visible to a network observer in TLS 1.2 and earlier), so a network controller attempting to confine the user's DNS traffic to a limited set of servers could use the ALPN extension as a signal to block DNS-specific streams. Another alternative could be an ALPN token that indicates potentially-multiplexed traffic (e.g. "http/1.1-or-dns"). This has a comparable problem when confronted with a network adversary that intends to penalize or hamper DNS-over-TLS. Existing HTTP clients will not send this token, and even if some start to offer it, it will provide less cover for DNS-over-TLS clients. Overview of initial octets ========================== DNS stream initial octets ------------------------- {{RFC1035}} section 4.2.2 ("TCP Usage") shows that every stream-based DNS connection starts with a DNS message, preceded with a 2-octet message length field: The message is prefixed with a two byte length field which gives the message length, excluding the two byte length field. {{RFC6895}} section 2 represents the DNS message header section, which is the first part of the DNS message on the wire (after the message length). 1 1 1 1 1 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | ID | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ |QR| OpCode |AA|TC|RD|RA| Z|AD|CD| RCODE | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | QDCOUNT/ZOCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | ANCOUNT/PRCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | NSCOUNT/UPCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | ARCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ So in a DNS over TCP stream, the interpretation of the initial 14 octets are fixed based on information about the first query sent on the stream: * 0,1: length of initial DNS message * 2,3: DNS Transaction ID * 4,5: DNS opcode, flags, and response code * 6,7: Question count (or Zone count in UPDATE) * 8,9: Answer count (or Prerequisite count in UPDATE) * 10,11: Authority count (or Update count in UPDATE) * 12,13: Additional RR count All DNS streams sent over TCP start with at least these 14 octets. HTTP/1.x initial octets ------------------- In an HTTP stream before HTTP/2, the first octets sent from the client are either the so-called `Simple-Request` (for HTTP/0.9) or the `Request-Line` (for HTTP/1.0 and HTTP/1.1). The data in this initial stream has variable characteristics. Most servers may wish to ignore the oldest of these, HTTP/0.9. ### HTTP/0.9 {{RFC1945}} section 4.1 says that HTTP/0.9 queries (that is, HTTP queries from before HTTP/1.0 was formalized) use this form: Simple-Request = "GET" SP Request-URI CRLF Note that HTTP/0.9 clients send this string and only this string, nothing else (no request body, no subsequent requests). The `Request-URI` token is guaranteed to start with a printable ASCII character, and cannot contain any members of the CTL class (values 0x00 through 0x1F) but due to loose early specifications, it might sometimes contain high-valued octets (those with the most-significant bit set -- 0x80 or above). So the first 5 octets are all constrained to be no less than 0x20 (SP) and no more than 0x7F (DEL), and all subsequent octets sent from the client have a value at least 0x0A (LF). The shortest possible HTTP/0.9 client request is: char: G E T SP / CR LF index: 0 1 2 3 4 5 6 The lowest possible HTTP/0.9 client request (sorted ASCIIbetically) is: char: G E T SP + : CR LF index: 0 1 2 3 4 5 6 7 ### HTTP/1.0 and HTTP/1.1 The request line format for HTTP/1.1 matches that of HTTP/1.0 (HTTP/1.1 adds protocol features like pipelining, but doesn't change the request form itself). But unlike HTTP/0.9, the initial verb (the `method`) can vary. {{RFC7230}} section 3.1.1 says that the first line of an HTTP/1.1 request is: request-line = method SP request-target SP HTTP-version CRLF method = token and {{RFC7230}} section 3.2.6 says: token = 1*tchar tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*" / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~" / DIGIT / ALPHA ; any VCHAR, except delimiters and VCHAR is defined in {{RFC5234}} appendix B.1 as: VCHAR = %x21-7E `request-target` itself cannot contain 0x20 (SP) or any CTL characters, or any characters above the US-ASCII range (> 0x7F). And the `HTTP-version` token is either the literal string "HTTP/1.0" or the literal string "HTTP/1.1", both of which are constrained to the same printable-ASCII range. The ASCIIbetically-lowest shortest possible HTTP/1.0 or HTTP/1.1 request is: char: ! SP / SP H T T P / 1 . 0 CR LF CR LF index: 0 1 2 3 4 5 6 7 8 9 a b c d e f In any case, no HTTP/1.0 or HTTP/1.1 request line can include any values lower than 0x0A (LF) or greater than 0x7F (DEL) in the first 16 octets. However, {{RFC7230}} section 3.1.1 also says: In the interest of robustness, a server that is expecting to receive and parse a request-line SHOULD ignore at least one empty line (CRLF) received prior to the request-line. So we should also consider accepting an arbitrary number of repeated CRLF sequences before the request-line as a potentially-valid HTTP client behavior. Specific octets =============== The sections below examine likely values of specific octet positions in the stream. All octet indexes are 0-based. octets 0 and 1 -------------- Any DNS message less than 3338 octets sent as the initial query over TCP can be reliably distinguished from any version of HTTP/1.x by the first two octets of the TCP stream alone. 3338 is 0x0D0A, or the ASCII string CRLF, which some HTTP/1.x clients might send before an initial request. No HTTP/1.x client can legitimately send anything lower than this. Most DNS queries are easily within this range automatically. octets 2 and 3 -------------- In a DNS stream, octets 2 and 3 represent the client-chosen message ID. The message ID is used to bind messages with responses. Over connectionless transports like UDP, this is an important anti-spoofing measure, as well as a distinguishing measure for clients reusing the same UDP port for multiple outstanding queries. Standard DNS clients already explicitly randomize this value. For the connection-oriented streaming DNS discussed here, the anti-spoofing characteristics are not relevant (the connection itself provides anti-spoofing), so the client is free to choose arbitrary values. With a standard DNS client which fully-randomizes these values, only 25% of generated queries will have the high bits of both octets set to 0. 100% of all HTTP/1.x requests will have the high bits of both of these octets cleared. Similarly, some small percentage of randomly-generated DNS queries will have values here lower than 0x0A, while no HTTP/1.x clients will ever send these low values. octet 4 ------- In a DNS stream, octet 4 combines several fields: 0 1 2 3 4 5 6 7 +--+--+--+--+--+--+--+--+ |QR| Opcode |AA|TC|RD| +--+--+--+--+--+--+--+--+ In a standard DNS query sent over a streaming interface, QR, Opcode, AA, and TC are all set to 0. The least-significant bit (RD -- Recursion Desired) is set when a packet is sent from a stub to a recursive resolver. The value of such an octet is 0x01. This value never occurs in octet 4 of a legitimate HTTP/1.x client. But under DNS UPDATE ({{RFC2136}}, Opcode is set to 5 and all the option bits are cleared, which means this value would have 0x40 (ASCII '@'), which could legitimately occur in some HTTP/1.x requests at this position. octet 5 ------- In a DNS stream, octet 5 also combines several fields: 0 1 2 3 4 5 6 7 +--+--+--+--+--+--+--+--+ |RA| Z|AD|CD| RCODE | +--+--+--+--+--+--+--+--+ In some DNS messages sent from a client, all these bits are 0. However, section 5.7 of {{RFC6840}} suggests that queries may wish to set the AD bit to indicate a desire to learn from a validating resolver whether the resolver considers the contents to be Authentic Data. {{RFC6840}} also suggests that: validating resolvers SHOULD set the CD bit on every upstream query. So many queries, particularly from DNSSEC-validating DNS clients, are likely to set bits 2 and 3, resulting in a value 0x30 (ASCII '0'). This is usually a legitimate value for octet 5 in an HTTP/1.x request. octets 6 and 7 -------------- In DNS, octets 6 and 7 represent the query count. Most DNS clients will send one query at a time, which makes this value 0x0001. As long as the number of initial queries does not exceed 0x0A0A (2570), then at least one of these octets will have a value less than 0x0A. No HTTP/1.x client sends an octet less than 0x0A in positions 6 or 7. In DNS UPDATE, octets 6 and 7 represent the zone count. Entries in the Zone section of the DNS UPDATE message are structured identically to entries in the Query section of a standard DNS message. octets 8 through 11 ------------------- In streaming DNS, octets 8 through 11 represent answer counts and authority counts in normal DNS queries, or Prerequisite and Update counts in DNS UPDATE. Standard DNS queries will set them both 0. DNS UPDATE queries are likely to include some records in these sections, so they won't be all zero, but as long as no more than 2570 Prerequisite records and no more than 2570 Update records are sent, at least one octet will have value less than 0x0A. But no HTTP/1.x client sends an octet less than 0x0A in these positions. octets 12 and 13 ---------------- In streaming DNS, octets 12 and 13 represent the number of Additional RRs. When a DNS query is sent with EDNS(0), the OPT RR is accounted for here. So this is often either 0x0000 or 0x0001. In a Secure DNS UPDATE {{RFC3007}}, the SIG(0) or TSIG record is also found in this section, which could increase the values of these octets to 0x0002. No HTTP/1.x client will send octets with these low values at these positions. Combinations of octets ====================== In a DNS message, each Question in the Question section (or Zone in the Zone section for DNS UPDATE) is at least 5 octets (1 octet for zero-length QNAME + 2 octets for QTYPE + 2 octets for QCLASS), and each RR (in the Answer, Authority, and Additional sections for normal DNS queries; or in the Prerequisite, Update, and Additional sections for DNS UPDATE) is at least 11 octets. And the header itself is 12 octets. So we know that for a valid DNS stream, the first message has a size of at least: min_first_msg_size = 12 + 5 * (256*o[6] + o[7]) + 11 * (256*(o[8] + o[10] + o[12]) + o[9] + o[11] + o[13]) It's possible to compare this value with the expected first query size: first_msg_size = 256 * o[0] + o[1] if `first_query_size` is less than `min_first_query_size` we can be confident that the stream is not DNS. ### Proof: a valid DNS message cannot be an HTTP/1.x query For any a valid, stream-based DNS message: * If there are fewer than 0x0A00 Questions then octet 6 < 0x0A. * If there are fewer than 0x0A00 Answer RRs, then octet 8 < 0x0A. * If there are fewer than 0x0A00 Authority RRs, then octet 10 < 0x0A. * If there are fewer than 0x0A00 Additional RRs, then octet 12 < 0x0A. If any of these four inequalities hold, then the packet is clearly DNS, not HTTP/1.x. if none of them hold, then there are at least 0x0A00 (2560) Questions and 3*2560 == 7680 RRs. But: 12 + 5*2560 + 11*7680 == 97292 So the smallest possible DNS message where none of these four inequalities hold is 97292 octets. But a DNS message is limited in size to 65535 octets. Therefore at least one of these inequalities holds, and one of the first 14 octets of a DNS steam is < 0x0A. But in a standard HTTP/1.x request, none of the first 14 octets can have a value < 0x0A, so a valid DNS message cannot be mistaken for an HTTP/1.x request. Guidance for Demultiplexing Servers =================================== Upon receiving a connection stream that might be either DNS or HTTP/1.x, a server can inspect the initial octets of the stream to decide where to send it. Without supporting HTTP/0.9 --------------------------- A server that doesn't care about HTTP/0.9 can simply wait for the first 14 octets of the client's request to come in. Then the algorithm is: bytestream = read_from_client(14) for x in bytestream: if (x < 0x0A) or (x > 0x7F): return `DNS` return `HTTP` Supporting archaic HTTP/0.9 clients ----------------------------------- A server that decides to try to support HTTP/0.9 clients has a slightly more challenging task, since some of them may send fewer octets than the initial DNS message, and the server shouldn't block waiting for data that will never come. bytestream = read_from_client(5) for x in bytestream[0:5] if (x < 0x0A) or (x > 0x7F): return `DNS` if (bytestream[0:4] != 'GET '): # not HTTP/0.9 bytestream += read_from_client(9) for x in bytestream[5:14]: if (x < 0x0A) or (x > 0x7f): return `DNS` return `HTTP` else: # maybe HTTP/0.9 seen_sp = False seen_high = False while (len(bytestream) < 14): if (seen_sp and seen_high): return `DNS` x = read_from_client(1) bytestream += x if (x > 0x7F): seen_high = True elif (x < 0x0A): return `DNS` elif (x == 0x20): seen_sp = True # SP found before CRLF, not HTTP/0.9 elif (x == 0x0A): return `HTTP` return `HTTP` Note that if read_from_client() ever fails to read the number of requested bytes (e.g. because of EOF), then the stream is neither valid HTTP nor valid DNS, and can be discarded. Signaling demultiplexing capacity --------------------------------- This document assumes that clients can learn out-of-band which listening service they can connect to. For example, the administrator of a machine can configure a local forwarding stub resolver to use DNS-over-TLS on port 443 of some specific server. This explicit configuration carries with it some level of trust -- the client is choosing to trust the configured server with its DNS queries. In some circumstances, it might be useful for a listener to signal to a client that it is willing and capable of handling both DNS and HTTP/1.x traffic. While such signalling could be useful for dynamic discovery, it opens questions of trust (which servers should the client be willing to rely on for DNS resolution?) and is out-of-scope for this draft. Guidance for DNS clients ======================== Consider a DNS client that connects to a server that might be interested in answering HTTP/1.x requests on the same address/port (or other channel identifier). The client wants to send traffic that is unambiguously DNS traffic to make it easy for the server to distinguish it from inbound HTTP/1.x requests. Fortunately, this is trivial to do. In fact, any sensibly-implemented DNS-over-TLS client can use this approach without modification, just by adjusting the port number of the upstream recursive resolver from 853 to 443. Such a client should follow these guidelines: * Send the DNS message size (a 16-bit integer) together in the same packet with the full header of the first DNS message so that the recipient can review as much as possible of the frame at once. This is a best practice for efficient stream-based DNS anyway. If the client is concerned about stream fragmentation that it cannot control, and it is talking to a server that might be expecting HTTP/0.9 clients, then the server might not be willing to wait for the full initial 14 octets to make a decision. Note that this fragmentation is not a concern for streams wrapped in TLS when using modern AEAD ciphersuites. In this case, the client gets to choose the size of the plaintext record, which is either recovered by the server in full (unfragmented) or the connection fails. If the client does not have such a guarantee from the transport, it MAY also take one of the following mitigating actions relating to the first DNS message it sends in the stream \[explanation of what the server gets to see in the fragmented stream case are in square brackets after each mitigation]: * Ensure the first message is marked as a query (QR = 0), and it uses opcode 0 ("Standard Query"). \[bytestream\[4] < 0x08] * Ensure that the first message has RA = 0, Z = 0, and RCODE = 0. \[bytestream\[5] == 0x00] * Ensure that the high bit of the first octet of the message ID of the first message is set. \[bytestream\[2] > 0x7F] * Send an initial short Server Status DNS message ahead of the otherwise intended initial DNS message. \[bytestream\[0] == 0x00] * Use the EDNS(0) padding option {{RFC7830}} to pad the first message to a multiple of 256 octets. \[bytestream\[1] == 0x00] Interpreting failure -------------------- FIXME: A DNS client that does not already know that a server is willing to carry both types of traffic SHOULD expect a transport connection failure of some sort. Can we say something specific about what it should expect? Guidance for HTTP clients ========================= HTTP clients SHOULD NOT send HTTP/0.9 requests, since modern HTTP servers are not required to support HTTP/0.9. Sending an HTTP/1.0 request (or any later version) is sufficient for a server to be able to distinguish the two protocols. Security Considerations ======================= FIXME: Clients should locally validate DNSSEC (servers may still be able to omit some records) FIXME: if widely deployed, consider amplification for DDoS against authoritative servers? FIXME: consider DNSSEC transparency FIXME: consider TLS session resumption -- this counts as a new stream boundary, so the multiplexing decision need not persist across resumption. FIXME: consider 0-RTT FIXME: consider X.509 cert validation FIXME: what other security considerations should clients take? FIXME: what other security considerations should servers take? Privacy Considerations ====================== FIXME: DNS queries and HTTP requests can reveal potentially sensitive information about the sender. FIXME: consider DNS and HTTP traffic analysis -- how should requests or responses be padded, aggregated, or delayed given that streams are multiplexed? FIXME: any other privacy considerations? IANA Considerations =================== This document does not ask IANA to make any changes to existing registries. However, it does update the DNS and HTTP specifications, to reflect the fact that services using this demultiplexing technique may be constrained in adoption of future versions of either stream-based DNS or HTTP/1.x if those future versions modify either protocol in a way that breaks with the distinctions documented here. In particular, this draft assumes that all future stream-based versions of HTTP/1.x should have the following properties: * the client will speak first * the client will send at least 14 octets before expecting a response from the server. * none of those first 14 octets will be below 0x0A (LF) or above 0x7F (DEL). Future extensions to stream-based DNS or HTTP/1.x should take this demultiplexing technique into consideration. Document Considerations ======================= \[ RFC Editor: please remove this section before publication ] This document is currently edited as markdown. Minor editorial changes can be suggested via merge requests at https://gitlab.com/dkg/hddemux or by e-mail to the author. Please direct all significant commentary to the public IETF DNS Privacy mailing list: dns-privacy@ietf.org or to the IETF HTTP WG mailing list: ietf-http-wg@w3.org hddemux-0.5/hddemux.1.md000066400000000000000000000060021417603527400151420ustar00rootroot00000000000000--- title: HDDEMUX section: 1 author: Daniel Kahn Gillmor date: 2017 May --- NAME ==== hddemux - demultiplexes incoming TCP connections between HTTP and DNS SYNOPSIS ======== hddemux DESCRIPTION =========== hddemux takes a set of listening stream-based file descriptors (see sd_listen_fds(3)) and accepts new connections on them. When a new connection comes in, it decides from the first few octets whether the connection is HTTP/1.x or DNS. If it thinks it's HTTP/1.x, it splices the connection to the HTTP_TARGET. If it thinks it's DNS, it splices the connection to the DNS_TARGET. ENVIRONMENT VARIABLES ===================== HTTP_TARGET and DNS_TARGET should be either a DNS name or an IP address, optionally followed by a colon and a port number. If either variable is unset, it will default to "localhost". If the port number is not specified, HTTP_TARGET defaults to TCP port 80, and DNS_TARGET defaults to TCP port 53. If HDDEMUX_DEBUG is set to a non-empty string, then it will send a lot of verbose debugging info to stderr. IDLE_TIMEOUT is used to set the number of milliseconds that hddemux will permit a session to be in an open state with no traffic passing before it tries to close both sides. The default is 600000 (10 minutes). Set this variable to 0 to never try to do this kind of cleanup. SIGNALS ======= When hddemux receives SIGUSR1, it dumps an overview of the current state of the demuxer (configuration, outstanding established streams, etc) to stderr. EXAMPLES ======== For systemd, you need a .socket unit file: #/lib/systemd/hddemux.socket [Unit] Description=HTTP/1.x and DNS demuxer socket Documentation=man:hddemux(1) [Socket] ListenStream=/run/hddemux/socket [Install] WantedBy=sockets.target and a .service unit file: #/lib/systemd/hddemux.service [Unit] Description=HTTP/1.x and DNS demuxer service Documentation=man:hddemux(1) Requires=hddemux.socket [Service] Type=notify ExecStart=/usr/bin/hddemux WorkingDirectory=/run/hddemux/workdir User=hddemux Group=hddemux [Install] Also=hddemux.socket Configuration can be done by overriding the .service file (e.g. with Service.Environment= entries). See the "edit" documentation in systemctl(1) and the "Overriding vendor settings" section of systemd.unit(5). CONSTRAINTS =========== hddemux is designed to demultiplex HTTP/1.x from stream-based DNS. Trying to demultiplex other protocols (including HTTP/2 or later) is not advised. Please see draft-dkg-dprive-demux-dns-http for more information and analysis. WARNING ======= Note that this effectively acts as a stream redirector once the client's first flight has been processed. It does this with no attempt to defend against stream redirection loops, so be careful not to redirect it to itself or it will eat all of your memory in infinite recursion. SEE ALSO ======== sd_listen_fds(3), systemctl(1), systemd.unit(5), https://datatracker.ietf.org/doc/draft-dkg-dprive-demux-dns-http/ hddemux-0.5/hddemux.c000066400000000000000000000675421417603527400146450ustar00rootroot00000000000000#include #include #include #include #include #include #include #include #include /* library includes */ #include #include #include #define INTERESTING_OCTETS 14 #define DEFAULT_IDLE_TIMEOUT (10*60*1000) enum protocol { DEMUX_HTTP = 0, DEMUX_DNS = 1, DEMUX_UNKNOWN = 2, }; const char *protocol_names[] = { "HTTP", "DNS", "UNKNNOWN" }; /* see README.md for justification of the heuristics below: */ static inline enum protocol surmise_protocol(unsigned char *bytestream, size_t len) { if (len < 5) return DEMUX_UNKNOWN; int ix; bool all_interesting = len >= INTERESTING_OCTETS; for (ix = 0; ix < 5; ix++) if ((bytestream[ix] < 0x0a) || (bytestream[ix] > 0x7f)) return DEMUX_DNS; if (memcmp(bytestream, "GET ", 4) != 0) { /* not HTTP/0.9 */ for (ix = 5; ix < (all_interesting ? INTERESTING_OCTETS : len); ix++) if ((bytestream[ix] < 0x0a) || (bytestream[ix] > 0x7f)) return DEMUX_DNS; } else { /* might be HTTP/0.9 */ bool seen_sp = false; bool seen_high = false; for (ix = 5; ix < (all_interesting ? INTERESTING_OCTETS : len); ix++) { if (seen_sp && seen_high) return DEMUX_DNS; if (bytestream[ix] > 0x7f) seen_high = true; else if (bytestream[ix] < 0x0a) return DEMUX_DNS; else if (bytestream[ix] == 0x20) seen_sp = true; else if (bytestream[ix] == 0x0a) return DEMUX_HTTP; } } if (all_interesting) return DEMUX_HTTP; else return DEMUX_UNKNOWN; } struct demuxer; /* managed streams, which will be sorted by last_touched */ struct streamstate { int id; struct { uv_tcp_t stream; size_t incoming; unsigned char *staging; size_t bytes_read; size_t bytes_written; uv_write_t writer; uv_shutdown_t shutdown; bool initialized; bool eof_seen; bool closed; } x[2]; size_t first_read_size; enum protocol surmised_protocol; uv_connect_t connect; bool released; uint64_t last_touched; uv_timer_t timer; bool timer_closed; }; int stream_splice(struct streamstate *s, uv_stream_t *from); int stream_exchange(struct streamstate *s); struct streamstate *new_streamstate(struct demuxer* demux); void free_streamstate(struct streamstate *str); void touch_streamstate(struct streamstate *st); /* program state: */ struct demuxer { struct addrinfo *http_targets, *dns_targets, *next_http, *next_dns; uv_loop_t *loop; uv_signal_t usr1_signal; uv_signal_t term_signal; int next_stream_id; int active_streams; bool debug; uint64_t stream_timeout; /* in milliseconds */ uv_tcp_t *listeners; int max_listeners; int listener_count; }; struct demuxer* new_demuxer(uv_loop_t *loop, int max_listeners); void free_demuxer(struct demuxer* demux); static void demuxer_rotate(struct demuxer *demuxer, enum protocol proto); static const struct addrinfo* demuxer_getaddrinfo(const struct demuxer *demuxer, enum protocol proto); static void demuxer_dump(struct demuxer *demuxer, FILE *f); void demuxer_signal_usr1(uv_signal_t *sig, int signum); void demuxer_signal_term(uv_signal_t *sig, int signum); /* workflow steps */ void new_inbound(uv_stream_t* server, int status); void first_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf); void inbound_first_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf); void stream_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf); void outbound_connection(uv_connect_t *req, int status); void write_complete(uv_write_t* write, int status); void stream_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf); void interesting_sent(uv_write_t* req, int status); void after_shutdown(uv_shutdown_t* req, int status); void close_streamstate_stream(uv_handle_t *stream); void close_streamstate_timer(uv_handle_t *timer); void streamstate_timeout(uv_timer_t *timer); struct min_max_streamstate { const struct streamstate *min, *max; }; void get_min_max_streamstate(uv_handle_t *h, void *x); /* *************** utilities *************** */ static void dump_addrinfo(FILE *f, const char *label, const struct addrinfo *a, bool is_current) { int port = 0; const char *fam; char paddr[INET6_ADDRSTRLEN]; const void *ia = NULL; if (a->ai_family == AF_INET6) { struct sockaddr_in6* ip = (struct sockaddr_in6*)a->ai_addr; port = ntohs(ip->sin6_port); fam = "AF_INET6"; ia = &(ip->sin6_addr); } else if (a->ai_family == AF_INET) { struct sockaddr_in* ip = (struct sockaddr_in*)a->ai_addr; port = ntohs(ip->sin_port); fam = "AF_INET"; ia = &(ip->sin_addr); } else { fam = "UNKNOWN"; } if (ia) { if (inet_ntop(a->ai_family, ia, paddr, sizeof(paddr)) == NULL) { int e = errno; /* these shenanigans are all for the GNU variant of strerror_r(3), not the XSI variant: */ const char * e2 = strerror_r(e, paddr+6, sizeof(paddr)-(6+1)); if (e2 != paddr+6) strncpy(paddr+6, e2, sizeof(paddr)-(6+1)); memcpy(paddr, " ", 6); paddr[sizeof(paddr)-1] = '\0'; } } else { strcpy(paddr, ""); } fprintf(f, "%s%s: fam: %s(%d) socktype: %d, proto: %d, addr: %s, port: %d\n", (is_current ? "[*]" : " "), label, fam, a->ai_family, a->ai_socktype, a->ai_protocol, paddr, port); } static int get_target_addrinfo(const char* envname, const char* defaultnode, const char *defaultservice, struct addrinfo**res, bool debug) { char *n = getenv(envname); const char *node = NULL, *service = NULL; const struct addrinfo hints = { .ai_flags = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; if (n != NULL) { char *colon = strrchr(n, ':'); char *bracket = strrchr(n, ']'); if (debug) fprintf(stderr, "target for %s is %s\n", envname, n); if (colon != NULL) { if (bracket != NULL && (bracket > colon)) { /* this was just a colon in a bracketed IPv6 address, not a port delimiter */ } else { *colon = '\0'; service = colon + 1; } } node = n; } else { if (debug) fprintf(stderr, "target for %s is default (%s)\n", envname, defaultnode); node = defaultnode; } if (service == NULL) { service = defaultservice; } return getaddrinfo(node, service, &hints, res); } /* *************** streamstate *************** */ void stream_dump(FILE *f, struct streamstate *s) { fprintf(f, "[%d](%p) in: %p out %p\n", s->id, (void*)s, (void*)&(s->x[0].stream), (void*)&(s->x[1].stream)); } struct streamstate *new_streamstate(struct demuxer *demux) { int err; struct streamstate *ret = calloc(1, sizeof(struct streamstate)); if (ret) { if ((err = uv_timer_init(demux->loop, &(ret->timer)))) { fprintf(stderr, "[%d] Failed to uv_timer_init: (%d) %s\n", ret->id, err, uv_strerror(err)); free(ret); return NULL; } ret->timer.data = ret; ret->surmised_protocol = DEMUX_UNKNOWN; ret->id = demux->next_stream_id++; demux->active_streams++; for (int ix = 0; ix < 2; ix++) { if ((err = uv_tcp_init(demux->loop, &(ret->x[ix].stream)))) { fprintf(stderr, "[%d] Failed to uv_tcp_init %s: (%d) %s\n", ret->id, (ix? "outbound":"inbound"), err, uv_strerror(err)); free(ret); return NULL; } ret->x[ix].stream.data = ret; } touch_streamstate(ret); } return ret; } void free_streamstate(struct streamstate *st) { if (!st->released) { st->released = true; for (int ix = 0; ix < 2; ix++) uv_close((uv_handle_t*) (&(st->x[ix].stream)), close_streamstate_stream); uv_close((uv_handle_t*)(&(st->timer)), close_streamstate_timer); } } void touch_streamstate(struct streamstate *st) { int err; struct demuxer *demux = st->x[0].stream.loop->data; st->last_touched = uv_now(demux->loop); if ((err = uv_timer_stop(&(st->timer)))) fprintf(stderr, "[%d] failed to uv_timer_stop: (%d) %s\n", st->id, err, uv_strerror(err)); if (demux->stream_timeout > 0) if ((err = uv_timer_start(&(st->timer), streamstate_timeout, demux->stream_timeout, 0))) fprintf(stderr, "[%d] failed to uv_timer_start(%llu): (%d) %s\n", st->id, (long long unsigned int)(demux->stream_timeout), err, uv_strerror(err)); } /* *************** demuxer *************** */ struct demuxer* new_demuxer(uv_loop_t *loop, int max_listeners) { int err; struct demuxer *ret = calloc(1, sizeof(struct demuxer)); if (ret == NULL) return NULL; ret->listeners = calloc(max_listeners, sizeof(uv_tcp_t)); if (ret->listeners == NULL) { free(ret); return NULL; } ret->max_listeners = max_listeners; ret->loop = loop; struct { uv_signal_t* sig; int signum; uv_signal_cb handler; } sigs[] = { { &(ret->usr1_signal), SIGUSR1, demuxer_signal_usr1 }, { &(ret->term_signal), SIGTERM, demuxer_signal_term } }; for (int ix = 0; ix < sizeof(sigs)/sizeof(sigs[0]); ix++) { if ((err = uv_signal_init(loop, sigs[ix].sig))) { fprintf(stderr, "Failed to init signal handler: (%d) %s\n", err, uv_strerror(err)); free(ret); return NULL; } if ((err = uv_signal_start(sigs[ix].sig, sigs[ix].handler, sigs[ix].signum))) { fprintf(stderr, "Failed to connect signal handler (%d): (%d) %s\n", sigs[ix].signum, err, uv_strerror(err)); free(ret); return NULL; } } loop->data = ret; return ret; } void demuxer_going_down(uv_handle_t *h, void *x) { if (h == NULL || x == NULL || h->data == NULL) return; struct demuxer *demux = x; if (h->loop != demux->loop) return; if (h->type != UV_TCP) return; struct streamstate *st = h->data; free_streamstate(st); } void free_demuxer(struct demuxer* demux) { uv_walk(demux->loop, demuxer_going_down, demux); for (int i = 0; i < demux->listener_count; i++) { uv_close((uv_handle_t*)(demux->listeners + i), NULL); } uv_close((uv_handle_t*)(&demux->usr1_signal), NULL); uv_close((uv_handle_t*)(&demux->term_signal), NULL); int err = uv_run(demux->loop, UV_RUN_NOWAIT); if (demux->debug) fprintf(stderr, "final loop instance completed: %d (%s)\n", err, err ? (err < 0 ? uv_strerror(err) : "unknown") : "none"); free(demux->listeners); free(demux); } static void demuxer_rotate(struct demuxer *demuxer, enum protocol proto) { if (proto == DEMUX_HTTP) { demuxer->next_http = demuxer->next_http->ai_next; if (demuxer->next_http == NULL) demuxer->next_http = demuxer->http_targets; } else if (proto == DEMUX_DNS) { demuxer->next_dns = demuxer->next_dns->ai_next; if (demuxer->next_dns == NULL) demuxer->next_dns = demuxer->dns_targets; } else { assert(false); } } static const struct addrinfo* demuxer_getaddrinfo(const struct demuxer *demuxer, enum protocol proto) { if (proto == DEMUX_HTTP) return demuxer->next_http; else if (proto == DEMUX_DNS) return demuxer->next_dns; else assert(false); } void demuxer_signal_usr1(uv_signal_t *sig, int signum) { assert(sig != NULL); assert(signum = SIGUSR1); struct demuxer *demux = sig->loop->data; demuxer_dump(demux, stderr); } void demuxer_signal_term(uv_signal_t *sig, int signum) { assert(sig != NULL); assert(signum = SIGTERM); uv_stop(sig->loop); } static inline void showtime(FILE *f, const char *label, int id, uint64_t t, uint64_t now) { assert(now >= t); fprintf(f, "%s: [%d] last_touched: %g seconds ago\n", label, id, (double)(now-t)/1000); } static void demuxer_dump(struct demuxer *demuxer, FILE *f) { const struct addrinfo *x; uint64_t now; now = uv_now(demuxer->loop); struct min_max_streamstate s = { .min = NULL, .max = NULL }; uv_walk(demuxer->loop, get_min_max_streamstate, &s); fprintf(f, "Total created: %d\nOutstanding: %d\nDefault timeout: %llums\n", demuxer->next_stream_id, demuxer->active_streams, (long long unsigned int)demuxer->stream_timeout); if (s.min) showtime(f, "oldest", s.min->id, s.min->last_touched, now); if (s.max) showtime(f, "youngest", s.max->id, s.max->last_touched, now); for (x = demuxer->dns_targets; x; x = x->ai_next) dump_addrinfo(f, "DNS", x, x == demuxer->next_dns); for (x = demuxer->http_targets; x; x = x->ai_next) dump_addrinfo(f, "HTTP", x, x == demuxer->next_http); } int demuxer_add_listener_fd(struct demuxer *demuxer, int fd, int listen_backlog) { int err; if (sd_is_socket(fd, AF_UNSPEC, SOCK_STREAM, 1) <= 0) { fprintf(stderr, "File descriptor %d is not a stream socket, ignoring.\n", fd); return 0; } if (demuxer->listener_count >= demuxer->max_listeners) { fprintf(stderr, "too many listeners! (have: %d, max: %d) ignoring file descriptor %d\n", demuxer->listener_count, demuxer->max_listeners, fd); return 0; } if ((err = uv_tcp_init(demuxer->loop, demuxer->listeners + demuxer->listener_count))) { fprintf(stderr, "failed to uv_tcp_init: %s\n", uv_strerror(err)); return 1; } /* listeners have NULL data, compared with normal streams, which point to their enclosing struct streamstate */ demuxer->listeners[demuxer->listener_count].data = NULL; if ((err = uv_tcp_open(demuxer->listeners + demuxer->listener_count, fd))) { fprintf(stderr, "Failed to uv_tcp_open (fd: %d): %s\n", fd, uv_strerror(err)); return 1; } if ((err = uv_listen((uv_stream_t*)(demuxer->listeners + demuxer->listener_count), listen_backlog, new_inbound))) { fprintf(stderr, "Failed to listen on stream (fd: %d): %s\n", fd, uv_strerror(err)); return 1; } demuxer->listener_count++; return 0; } /* *************** workflow stages *************** */ void new_inbound(uv_stream_t* server, int status) { assert(server != NULL); if (status) { fprintf(stderr, "ignoring weird non-zero new_inbound() status: (%d) %s\n", status, uv_strerror(status)); } else { struct streamstate *st = new_streamstate(server->loop->data); int err; if (st == NULL) { fprintf(stderr, "No RAM for new connection\n"); return; } if ((err = uv_accept(server, (uv_stream_t*)(&(st->x[0].stream))))) { fprintf(stderr, "failed to uv_accept: %s\n", uv_strerror(err)); free_streamstate(st); return; } st->x[0].initialized = true; if ((err = uv_read_start((uv_stream_t*)(&(st->x[0].stream)), first_alloc, inbound_first_read))) { fprintf(stderr, "failed to uv_read_start: %s\n", uv_strerror(err)); free_streamstate(st); return; } } } void first_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf) { struct streamstate *st = handle->data; assert(st != NULL); assert((uv_handle_t*)(&st->x[0].stream) == handle); if (st->x[0].staging == NULL) { if (suggested_size < INTERESTING_OCTETS) suggested_size = INTERESTING_OCTETS; st->x[0].staging = malloc(suggested_size); if (st->x[0].staging == NULL) fprintf(stderr, "first_alloc malloc() failed: (%d) %s\n", errno, strerror(errno)); st->first_read_size = suggested_size; } buf->base = (char*)(st->x[0].staging) + st->x[0].bytes_read; if (buf->base) buf->len = st->first_read_size - st->x[0].bytes_read; else buf->len = 0; } void inbound_first_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf) { int err; assert(stream != NULL); assert(stream->data != NULL); struct streamstate *st = stream->data; assert((uv_stream_t*)(&(st->x[0].stream)) == stream); /* this must only be called on input! */ assert(st->x[0].initialized); assert(st->surmised_protocol == DEMUX_UNKNOWN); assert((unsigned char*)(buf->base) == st->x[0].staging + st->x[0].bytes_read); struct demuxer *demuxer = stream->loop->data; if (nread == 0 || nread == UV_EAGAIN || nread == UV_EBUSY) { fprintf(stderr, "[%d] ignoring weird inbound_first_read() with status: (%zd), %s\n", st->id, nread, uv_strerror(nread)); return; } if (nread < 0) { fprintf(stderr, "[%d] error on inbound stream before we could interpret it (%zd): %s\n", st->id, nread, uv_strerror(nread)); free_streamstate(st); return; } if (demuxer->debug) fprintf(stderr, "[%d] inbound initial read %zd octets\n", st->id, nread); st->x[0].bytes_read += nread; st->surmised_protocol = surmise_protocol(st->x[0].staging, st->x[0].bytes_read); if (st->surmised_protocol == DEMUX_UNKNOWN) { fprintf(stderr, "[%d] inbound stream is still unknown: only sent %zd octets so far (%zd in this flight)\n", st->id, st->x[0].bytes_read, nread); touch_streamstate(st); return; } /* do not read more from the inbound stream until we've established the outbound stream */ if ((err = uv_read_stop(stream))) { fprintf(stderr, "[%d] inbound initial uv_read_stop error %d: %s\n", st->id, err, uv_strerror(err)); free_streamstate(st); return; } const struct addrinfo *addr = demuxer_getaddrinfo(demuxer, st->surmised_protocol); if (demuxer->debug) { fprintf(stderr, "[%d] new stream is %s\n", st->id, protocol_names[st->surmised_protocol]); dump_addrinfo(stderr, protocol_names[st->surmised_protocol], addr, false); } demuxer_rotate(demuxer, st->surmised_protocol); if ((err = uv_tcp_connect(&st->connect, &(st->x[1].stream), addr->ai_addr, outbound_connection))) { fprintf(stderr, "[%d] Failed to uv_tcp_connect during new outbound stream: (%d) %s\n", st->id, err, uv_strerror(err)); free_streamstate(st); } touch_streamstate(st); } void outbound_connection(uv_connect_t *req, int status) { int err; assert(req != NULL); assert(req->handle != NULL); struct streamstate *st = req->handle->data; assert(st != NULL); assert(((uv_stream_t*)(&(st->x[1]))) == req->handle); /* this should be the outbound leg */ assert(!st->x[1].initialized); if (status) { fprintf(stderr, "[%d] failed to complete the connection: status %d (%s)\n", st->id, status, uv_strerror(status)); free_streamstate(st); return; } st->x[1].initialized = true; /* send the interesting bits along */ const struct uv_buf_t buf = { .base = (char*)(st->x[0].staging), .len = st->x[0].bytes_read }; if ((err = uv_write(&(st->x[1].writer), (uv_stream_t*)(&(st->x[1].stream)), &buf, 1, write_complete))) { fprintf(stderr, "[%d] Failed to uv_write first frame to outbound: (%d) %s\n", st->id, err, uv_strerror(err)); free_streamstate(st); return; } /* start reading from outbound: */ if ((err = uv_read_start((uv_stream_t*)(&(st->x[1].stream)), stream_alloc, stream_read))) { fprintf(stderr, "[%d] Failed to uv_read_start outbound: (%d) %s\n", st->id, err, uv_strerror(err)); free_streamstate(st); return; } touch_streamstate(st); } void stream_alloc(uv_handle_t* handle, size_t suggested_size, uv_buf_t* buf) { struct streamstate *st = handle->data; assert(st != NULL); for (int ix = 0; ix < 2; ix++) if ((uv_tcp_t*)handle == &(st->x[ix].stream)) { assert(st->x[ix].staging == NULL); st->x[ix].staging = malloc(suggested_size); if (st->x[ix].staging == NULL) fprintf(stderr, "stream_alloc malloc() failed: (%d) %s\n", errno, strerror(errno)); buf->base = (char*)(st->x[ix].staging); buf->len = buf->base ? suggested_size : 0; } } void write_complete(uv_write_t* write, int status) { int err; assert(write != NULL); assert(write->handle != NULL); struct streamstate *st = write->handle->data; assert(st != NULL); assert(st->x[0].initialized); assert(st->x[1].initialized); for (int ix = 0; ix < 2; ix++) if ((uv_tcp_t*)(write->handle) == &(st->x[ix].stream)) { /* free the buffer from the other since it has been written: */ assert(st->x[!ix].staging != NULL); free(st->x[!ix].staging); st->x[!ix].staging = NULL; if (status) { fprintf(stderr, "[%d] write to %s failed: (%d) %s\n", st->id, (ix?"outbound":"inbound"), status, uv_strerror(status)); free_streamstate(st); return; } /* re-enable reading from the other */ if ((err = uv_read_start((uv_stream_t*)&(st->x[!ix].stream), stream_alloc, stream_read))) { fprintf(stderr, "[%d] re-enabling reading from %s failed: (%d) %s\n", st->id, (!ix?"outbound":"inbound"), status, uv_strerror(status)); free_streamstate(st); return; } touch_streamstate(st); return; } assert(false); /* should never reach here */ } void stream_read(uv_stream_t* stream, ssize_t nread, const uv_buf_t* buf) { assert(stream != NULL); struct streamstate *st = stream->data; assert(st != NULL); assert(st->x[0].initialized); assert(st->x[1].initialized); struct demuxer *demuxer = stream->loop->data; assert(demuxer != NULL); int err; for (int ix = 0; ix < 2; ix++) if ((uv_tcp_t*)stream == &(st->x[ix].stream)) { const char *indir = (ix? "outbound" : "inbound"); const char *outdir = (ix? "inbound" : "outbound"); assert(!st->x[ix].eof_seen); touch_streamstate(st); if (nread == 0 || nread == UV_EAGAIN || nread == UV_EBUSY || nread ==UV_ENOBUFS || nread == UV_EINTR) { fprintf(stderr, "[%d] ignoring weird stream_read() with status: (%zd), %s\n", st->id, nread, uv_strerror(nread)); return; } if (nread == UV_EOF) { if (demuxer->debug) fprintf(stderr, "[%d] EOF received from %s leg\n", st->id, indir); /* we've just seen the EOF from one channel -- shut down the write side of the other. */ st->x[ix].eof_seen = true; if ((err = uv_shutdown(&(st->x[!ix].shutdown), (uv_stream_t*)(&(st->x[!ix].stream)), after_shutdown))) { fprintf(stderr, "[%d] error calling uv_shutdown() on %s leg: (%d) %s\n", st->id, outdir, err, uv_strerror(err)); free_streamstate(st); } return; } if (nread < 0) { fprintf(stderr, "[%d] closing due to %s read error: (%zd): %s\n", st->id, indir, nread, uv_strerror(nread)); free_streamstate(st); return; } if (demuxer->debug) fprintf(stderr, "[%d] %s read %zd octets\n", st->id, indir, nread); assert((unsigned char*)(buf->base) == st->x[ix].staging); if ((err = uv_read_stop(stream))) { fprintf(stderr, "[%d] %s uv_read_stop error %d: %s\n", st->id, indir, err, uv_strerror(err)); free_streamstate(st); return; } st->x[ix].bytes_read += nread; const struct uv_buf_t buf = { .base = (char*)(st->x[ix].staging), .len = nread }; /* schedule write to other channel */ if ((err = uv_write(&(st->x[!ix].writer), (uv_stream_t*)&(st->x[!ix].stream), &buf, 1, write_complete))) { fprintf(stderr, "[%d] %s uv_write failed: (%d) %s\n", st->id, outdir, err, uv_strerror(err)); free_streamstate(st); return; } return; } assert(false); /* should never reach here */ } void after_shutdown(uv_shutdown_t* req, int status) { assert(req != NULL); assert(req->handle != NULL); assert(req->handle->data != NULL); struct streamstate *st = req->handle->data; assert(st->x[0].initialized); assert(st->x[1].initialized); for (int ix = 0; ix < 2; ix++) if ((uv_tcp_t*)(req->handle) == &(st->x[ix].stream)) { if (status) { fprintf(stderr, "[%d] error in shutdown of %s stream: (%d) %s\n", st->id, (ix?"outbound":"inbound"), status, uv_strerror(status)); free_streamstate(st); return; } /* if we've seen eof on both sides, we can shut this down */ if (st->x[!ix].eof_seen) free_streamstate(st); else touch_streamstate(st); return; } assert(false); /* should never have gotten here. */ } inline void streamstate_dispose(struct streamstate *st, struct demuxer *demux) { if (st->timer_closed && st->x[0].closed && st->x[1].closed) { demux->active_streams--; for (int ix = 0; ix < 2; ix++) if (st->x[ix].staging) { free(st->x[ix].staging); st->x[ix].staging = NULL; } free(st); } } void close_streamstate_timer(uv_handle_t *timer) { assert(timer != NULL); assert(timer->data != NULL); struct streamstate *st = timer->data; assert(timer->loop != NULL); struct demuxer *demux = timer->loop->data; if (demux->debug) fprintf(stderr, "[%d] close_streamstate_timer\n", st->id); st->timer_closed = true; streamstate_dispose(st, demux); } void close_streamstate_stream(uv_handle_t *stream) { assert(stream != NULL); assert(stream->data != NULL); struct streamstate *st = stream->data; int found = -1; assert(stream->loop != NULL); struct demuxer *demux = stream->loop->data; assert(demux != NULL); for (int ix = 0; ix < 2; ix++) if ((uv_tcp_t*)stream == &(st->x[ix].stream)) { found = ix; } if (demux->debug) fprintf(stderr, "[%d] close_streamstate_stream %d\n", st->id, found); assert(found != -1); st->x[found].closed = true; streamstate_dispose(st, demux); } void streamstate_timeout(uv_timer_t *timer) { assert(timer != NULL); assert(timer->data != NULL); struct streamstate *st = timer->data; assert(timer->loop != NULL); struct demuxer *demux = timer->loop->data; assert(demux != NULL); if (demux->debug) fprintf(stderr, "[%d] closing stream due to IDLE_TIMEOUT\n", st->id); free_streamstate(st); } void get_min_max_streamstate(uv_handle_t *h, void *x) { if (h->type != UV_TCP) return; if (h->data == NULL) return; if (h->data == h->loop) return; if (x == NULL) return; struct min_max_streamstate *minmax = x; struct streamstate *s = h->data; if (minmax->min == NULL || minmax->min->last_touched > s->last_touched) minmax->min = s; if (minmax->max == NULL || minmax->max->last_touched < s->last_touched) minmax->max = s; } int main(int argc, const char **argv) { int lmax = 0; int err; uv_loop_t loop; int listen_backlog = 10; struct demuxer *demuxer = NULL; if ((err = uv_loop_init(&loop)) < 0) { fprintf(stderr, "Failed to initialize uv_loop: %s\n", uv_strerror(err)); return 1; } lmax = sd_listen_fds(0); demuxer = new_demuxer(&loop, lmax); if ((err = setvbuf(stderr, NULL, _IONBF, 0))) { fprintf(stderr, "Failed to unbuffer stderr: %s\n", strerror(errno)); return 1; } for (int i = 0; i < lmax; i++) { int fd = SD_LISTEN_FDS_START + i; if ((err = demuxer_add_listener_fd(demuxer, fd, listen_backlog))) { fprintf(stderr, "Failed to listen on fd %d: (%d) %s\n", fd, err, uv_strerror(err)); return 1; } } if (demuxer->listener_count < 1) { fprintf(stderr, "No listening sockets available! nothing to do.\n"); return 1; } char *debug = getenv("HDDEMUX_DEBUG"); demuxer->debug = (debug != NULL && debug[0] != '\0'); char *timeout = getenv("IDLE_TIMEOUT"); if (timeout) demuxer->stream_timeout = strtoull(timeout, NULL, 0); else demuxer->stream_timeout = DEFAULT_IDLE_TIMEOUT; if ((err = get_target_addrinfo("HTTP_TARGET", "localhost", "80", &demuxer->http_targets, demuxer->debug))) { fprintf(stderr, "Failed looking up HTTP_TARGET: %s\n", gai_strerror(err)); return 1; } if ((err = get_target_addrinfo("DNS_TARGET", "localhost", "53", &demuxer->dns_targets, demuxer->debug))) { fprintf(stderr, "Failed looking up DNS_TARGET: %s\n", gai_strerror(err)); return 1; } demuxer->next_http = demuxer->http_targets; demuxer->next_dns = demuxer->dns_targets; if (demuxer->debug) demuxer_dump(demuxer, stderr); sd_notify(0, "READY=1"); err = uv_run(&loop, UV_RUN_DEFAULT); if (demuxer->debug) fprintf(stderr, "loop completed: %d (%s)\n", err, err ? (err < 0 ? uv_strerror(err) : "unknown") : "none"); freeaddrinfo(demuxer->http_targets); freeaddrinfo(demuxer->dns_targets); free_demuxer(demuxer); if ((err = uv_loop_close(&loop))) fprintf(stderr, "uv_loop_close() failed: (%d) %s\n", err, uv_strerror(err)); return 0; } hddemux-0.5/hddemux.conf000066400000000000000000000001751417603527400153350ustar00rootroot00000000000000# tmpfiles.d(5) runtime directory for hddemux d /run/hddemux 0751 root root - - d /run/hddemux/workdir 0750 root hddemux - - hddemux-0.5/hddemux.service000066400000000000000000000003731417603527400160500ustar00rootroot00000000000000[Unit] Description=HTTP/1.x and DNS demuxer service Documentation=man:hddemux(1) Requires=hddemux.socket [Service] Type=notify ExecStart=/usr/bin/hddemux WorkingDirectory=/run/hddemux/workdir User=hddemux Group=hddemux [Install] Also=hddemux.socket hddemux-0.5/hddemux.socket000066400000000000000000000002361417603527400156760ustar00rootroot00000000000000[Unit] Description=HTTP/1.x and DNS demuxer socket Documentation=man:hddemux(1) [Socket] ListenStream=/run/hddemux/socket [Install] WantedBy=sockets.target hddemux-0.5/testsuite000077500000000000000000000106761417603527400150160ustar00rootroot00000000000000#!/bin/bash # test suite for hddemux # requires: # - nginx # - knot-resolver # - kdig (from knot-dnsutils) # - curl # - certtool (from gnutls-bin) # environment variables: # - WORKDIR: a place for all generated files. # if unset, it will be auto-generated. # it will be created as needed. # if the directory doesn't currently exist, it will be cleaned up at exit. # if it already exists, it will not be cleaned up. # - TESTIP: the IP address to use for testing. # the user needs to be able to open listening sockets, and to connect to them # by default, choose a random IP in 127.0.0.0/8 # Author: Daniel Kahn Gillmor # 2018-08-29 # License: GPLv3+ # error on exit set -e # for handling jobspecs: set -m # Unset proxy to make sure curl behaves correctly unset https_proxy http_proxy hddemux=$(which hddemux) || hddemux=./hddemux [ -x "$hddemux" ] if [ -z "$WORKDIR" ]; then d="$(mktemp -d)" remove="$d" else d="$WORKDIR" fi ip="${TESTIP:-127.$(( $RANDOM % 256 )).$(( $RANDOM % 256 )).$(( $RANDOM % 256 ))}" printf "hddemux test\n------------\n binary: %s\n workdir: %s\n IP addr: %s\n" "$hddemux" "$d" "$ip" section() { printf "\n%s\n" "$1" sed 's/./-/g' <<<"$1" } cleanup () { section "cleaning up" find "$d" -ls tail "$d/"*.err /usr/sbin/nginx -c "$d/nginx.conf" -p "$d" -s stop 2> "$d/nginx-stop.err" || true kill %2 || true kill %1 || true if [ "$remove" ]; then printf "cleaning up working directory %s\n" "$remove" rm -rf "$remove" fi } trap cleanup EXIT section "simple failing run" # hddemux with no arguments and no listening file descriptors should fail: if "$hddemux" 2>&1; then false fi section "make Certificate Authority key and certificate" cat > "$d/ca.template" < "$d/ee.template" < "$d/kresd.conf" < iterate' } net.tls("$d/ee-cert.pem", "$d/ee-key.pem") hints["monkeys.example"] = "127.15.23.5" EOF /usr/sbin/kresd --config "$d/kresd.conf" --tls "$ip@8853" --noninteractive "$d" 2> "$d/kresd.err" & section "make hddemux configuration on $ip:2000" systemd-socket-activate -l "$ip:2000" -E HTTP_TARGET="$ip:8853" -E DNS_TARGET="$ip:8853" "$hddemux" 2> "$d/hddemux.err" & section "set up nginx on $ip:4433" mkdir -p "$d/nginx" cat >"$d/nginx.conf" < "$d/data/index.txt" /usr/sbin/nginx -c "$d/nginx.conf" -p "$d" 2> "$d/nginx.err" section "test with kdig" x=$(kdig +short +tls +tls-ca="$d/ca-cert.pem" +tls-hostname=test.example @"$ip:2000" monkeys.example) [ "$x" = "127.15.23.5" ] echo "successful DNS-over-TLS request to $ip on port 2000" section "test with curl" x=$(curl --silent --show-error --cacert "$d/ca-cert.pem" --resolve "test.example:2000:$ip" --resolve "test.example:4433:$ip" https://test.example:4433/) [ "$x" = "Hello, world!" ] echo "successful HTTPS request to $ip on port 2000"