NNTP C. Feather Internet-Draft Thus plc Expires: August 30, 2003 March 1, 2003 Network News Transport Protocol draft-ietf-nntpext-base-17 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 30, 2003. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract The Network News Transport Protocol has been in use in the Internet for a decade and remains one of the most popular protocols (by volume) in use today. This document is a replacement for RFC 977 and officially updates the protocol specification. It clarifies some vagueness in RFC 977, includes some new base functionality and provides a specific mechanism to add standardized extensions to NNTP. Administration This document is a product of the NNTP Working Group, chaired by Russ Allbery. Feather Expires August 30, 2003 [Page 1] Internet-Draft Network News Transport Protocol March 2003 This is draft 17 pre-publication version 2. Outstanding issues Outstanding substantive (as opposed to editorial) issues in the text are shown as in the following case. OUTSTANDING ISSUE Reference consistency: should every RFC that is mentioned be included in the references? Where the same document is referred to in more than one place, should every occasion have a reference number (that is, "RFC 977 [3]" or similar), or only the first one, or only the first one in each section? Author's Note This draft is the first produced using a new formatting process. It therefore may contain unintentional layout or formatting changes compared with previous drafts. The author would appreciate being informed of any problems this has caused. This draft is written in XML using an NNTP-specific DTD. Custom software is used to convert this to RFC 2629 [12] format, and then the public "xml2rfc" package to further reduce this to text, nroff source, and HTML. No perl was used in producing this draft. Rights UNIX is a registered trademark of the X/Open Company Ltd. Feather Expires August 30, 2003 [Page 2] Internet-Draft Network News Transport Protocol March 2003 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . 7 2. Notation . . . . . . . . . . . . . . . . . . . . . . . . 8 3. Basic Operation . . . . . . . . . . . . . . . . . . . . 9 3.1 Response Codes . . . . . . . . . . . . . . . . . . . . . 11 3.1.1 Generic Response Codes . . . . . . . . . . . . . . . . . 13 3.1.1.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2 Pipelining . . . . . . . . . . . . . . . . . . . . . . . 16 3.2.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . 17 4. The WILDMAT format . . . . . . . . . . . . . . . . . . . 18 4.1 Wildmat syntax . . . . . . . . . . . . . . . . . . . . . 18 4.2 Wildmat semantics . . . . . . . . . . . . . . . . . . . 18 4.3 Extensions . . . . . . . . . . . . . . . . . . . . . . . 19 4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . 20 5. The GREETING Step . . . . . . . . . . . . . . . . . . . 21 5.1 Initial Connection . . . . . . . . . . . . . . . . . . . 21 5.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 21 5.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 21 5.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 21 5.2 MODE READER . . . . . . . . . . . . . . . . . . . . . . 22 5.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 22 5.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 22 5.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 23 6. The CAPABILITIES DISCOVERY step . . . . . . . . . . . . 25 6.1 LIST EXTENSIONS . . . . . . . . . . . . . . . . . . . . 25 6.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 25 6.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 26 7. Article posting and retrieval . . . . . . . . . . . . . 27 7.1 Group and article selection . . . . . . . . . . . . . . 27 7.1.1 GROUP . . . . . . . . . . . . . . . . . . . . . . . . . 27 7.1.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 28 7.1.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 28 7.1.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 29 7.1.2 LAST . . . . . . . . . . . . . . . . . . . . . . . . . . 30 7.1.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 30 7.1.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 30 7.1.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 31 7.1.3 NEXT . . . . . . . . . . . . . . . . . . . . . . . . . . 32 7.1.3.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 32 7.1.3.2 Description . . . . . . . . . . . . . . . . . . . . . . 32 7.1.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 32 7.2 Retrieval of articles and article sections . . . . . . . 33 7.2.1 ARTICLE . . . . . . . . . . . . . . . . . . . . . . . . 33 7.2.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 33 7.2.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 34 7.2.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 35 Feather Expires August 30, 2003 [Page 3] Internet-Draft Network News Transport Protocol March 2003 7.2.2 HEAD . . . . . . . . . . . . . . . . . . . . . . . . . . 36 7.2.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 36 7.2.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 37 7.2.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 37 7.2.3 BODY . . . . . . . . . . . . . . . . . . . . . . . . . . 39 7.2.3.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 39 7.2.3.2 Description . . . . . . . . . . . . . . . . . . . . . . 39 7.2.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 39 7.2.4 STAT . . . . . . . . . . . . . . . . . . . . . . . . . . 40 7.2.4.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 40 7.2.4.2 Description . . . . . . . . . . . . . . . . . . . . . . 41 7.2.4.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 41 7.3 Article posting . . . . . . . . . . . . . . . . . . . . 42 7.3.1 POST . . . . . . . . . . . . . . . . . . . . . . . . . . 42 7.3.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 42 7.3.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 43 7.3.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 44 7.3.2 IHAVE . . . . . . . . . . . . . . . . . . . . . . . . . 44 7.3.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 44 7.3.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 45 7.3.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 46 8. Information commands . . . . . . . . . . . . . . . . . . 48 8.1 DATE . . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 48 8.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 48 8.2 HELP . . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 49 8.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 49 8.3 NEWGROUPS . . . . . . . . . . . . . . . . . . . . . . . 49 8.3.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 49 8.3.2 Description . . . . . . . . . . . . . . . . . . . . . . 49 8.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 50 8.4 NEWNEWS . . . . . . . . . . . . . . . . . . . . . . . . 50 8.4.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 51 8.4.2 Description . . . . . . . . . . . . . . . . . . . . . . 51 8.4.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 51 8.5 Time . . . . . . . . . . . . . . . . . . . . . . . . . . 52 8.5.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . 52 8.6 The LIST commands . . . . . . . . . . . . . . . . . . . 53 8.6.1 LIST ACTIVE . . . . . . . . . . . . . . . . . . . . . . 53 8.6.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 53 8.6.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 53 8.6.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 54 8.6.2 LIST ACTIVE.TIMES . . . . . . . . . . . . . . . . . . . 55 8.6.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 55 8.6.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 55 Feather Expires August 30, 2003 [Page 4] Internet-Draft Network News Transport Protocol March 2003 8.6.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 55 8.6.3 LIST DISTRIBUTIONS . . . . . . . . . . . . . . . . . . . 56 8.6.3.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 56 8.6.3.2 Description . . . . . . . . . . . . . . . . . . . . . . 56 8.6.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 57 8.6.4 LIST DISTRIB.PATS . . . . . . . . . . . . . . . . . . . 57 8.6.4.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 57 8.6.4.2 Description . . . . . . . . . . . . . . . . . . . . . . 57 8.6.4.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 58 8.6.5 LIST NEWSGROUPS . . . . . . . . . . . . . . . . . . . . 58 8.6.5.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 58 8.6.5.2 Description . . . . . . . . . . . . . . . . . . . . . . 59 8.6.5.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 59 9. The CONCLUSION step . . . . . . . . . . . . . . . . . . 60 9.1 QUIT . . . . . . . . . . . . . . . . . . . . . . . . . . 60 9.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 60 9.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 60 9.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 60 10. Framework for NNTP extensions . . . . . . . . . . . . . 61 10.1 Initial IANA registry . . . . . . . . . . . . . . . . . 63 10.2 Standard extensions . . . . . . . . . . . . . . . . . . 63 10.3 The LISTGROUP extension . . . . . . . . . . . . . . . . 63 10.3.1 LISTGROUP . . . . . . . . . . . . . . . . . . . . . . . 63 10.3.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 63 10.3.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 64 10.3.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 64 10.4 Article metadata . . . . . . . . . . . . . . . . . . . . 65 10.4.1 The :bytes metadata item . . . . . . . . . . . . . . . . 65 10.4.2 The :lines metadata item . . . . . . . . . . . . . . . . 66 10.5 The OVER extension . . . . . . . . . . . . . . . . . . . 66 10.5.1 OVER . . . . . . . . . . . . . . . . . . . . . . . . . . 66 10.5.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 66 10.5.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 67 10.5.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 68 10.5.2 LIST OVERVIEW.FMT . . . . . . . . . . . . . . . . . . . 69 10.5.2.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 69 10.5.2.2 Description . . . . . . . . . . . . . . . . . . . . . . 70 10.5.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 71 10.6 The HDR extension . . . . . . . . . . . . . . . . . . . 72 10.6.1 HDR . . . . . . . . . . . . . . . . . . . . . . . . . . 72 10.6.1.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . 72 10.6.1.2 Description . . . . . . . . . . . . . . . . . . . . . . 72 10.6.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . 74 11. Augmented BNF Syntax for NNTP Commands . . . . . . . . . 76 12. Security Considerations . . . . . . . . . . . . . . . . 79 12.1 Personal and Proprietary Information . . . . . . . . . . 79 12.2 Abuse of Server Log Information . . . . . . . . . . . . 79 12.3 Weak Authentication and Access Control . . . . . . . . . 79 Feather Expires August 30, 2003 [Page 5] Internet-Draft Network News Transport Protocol March 2003 12.4 DNS Spoofing . . . . . . . . . . . . . . . . . . . . . . 80 12.5 UTF-8 issues . . . . . . . . . . . . . . . . . . . . . . 80 13. Acknowledgments . . . . . . . . . . . . . . . . . . . . 82 Normative References . . . . . . . . . . . . . . . . . . 84 Informative References . . . . . . . . . . . . . . . . . 85 Author's Address . . . . . . . . . . . . . . . . . . . . 85 Intellectual Property and Copyright Statements . . . . . 86 Feather Expires August 30, 2003 [Page 6] Internet-Draft Network News Transport Protocol March 2003 1. Introduction This document specifies the Network News Transport Protocol (NNTP), which is used for the distribution, inquiry, retrieval, and posting of net news articles using a reliable stream-based mechanism. For news reading clients, NNTP enables retrieval of news articles that are stored in a central database, giving subscribers the ability to select only those articles they wish to read. The net news model provides for indexing, cross-referencing, and expiration of aged messages. For server-to-server interaction, NNTP is designed for efficient transmission of net news articles over a reliable full duplex communication channel. Every attempt is made to ensure that the protocol specification in this document is compatible with the version specified in RFC 977 [1]. However, this version does not support the ill-defined SLAVE command and permits four digit years to be specified in the NEWNEWS and NEWGROUPS commands. It changes the default character set to UTF-8 [2] instead of US-ASCII [3]. It also extends the newsgroup name matching capabilities already documented in RFC 977. Generally, new functionality is made available using new commands. Part of that new functionality involves a mechanism to discover what new functionality is available to clients from a server. This mechanism can also be used to add more functionality as needs merit such additions. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [4]. An implementation is not compliant if it fails to satisfy one or more of the MUST requirements for this protocol. An implementation that satisfies all the MUST and all the SHOULD requirements for its protocols is said to be "unconditionally compliant"; one that satisfies all the MUST requirements but not all the SHOULD requirements for NNTP is said to be "conditionally compliant". For the remainder of this document, the term "client host" refers to a host making use of the NNTP service, while the term "server host" refers to a host that offers the NNTP service. Feather Expires August 30, 2003 [Page 7] Internet-Draft Network News Transport Protocol March 2003 2. Notation The following notational conventions are used in this document. UPPERCASE indicates literal text to be included in the command; lowercase indicates a token described elsewhere; [brackets] indicate that the parameter is optional; ellipsis... indicates that the parameter may be repeated any number of times (it must occur at least once); vertical|bar indicates a choice of two mutually exclusive parameters (exactly one must be provided). The name "message-id" for a command or response parameter indicates that it is the message-id of an article as described in Section 7. The actual parameter MUST include the angle brackets. The name "wildmat" for a parameter indicates that it is a wildmat as defined in Section 4. If the parameter does not meet the requirements of that section (for example, if it does not fit the grammar of Section 4.1) the NNTP server MAY place some interpretation on it (not specified by this document) or otherwise MUST treat it as a syntax error. Responses for each command will be described in tables listing the required format of a response followed by the meaning that should be ascribed to that response. Examples in this document are not normative but serve to illustrate usages, arguments, and responses. In the examples, a "[C]" will be used to represent the client host and a "[S]" will be used to represent the server host. Most of the examples do not rely on a particular server state. In some cases, however, they do assume that the current selected newsgroup (see the GROUP command (Section 7.1.1)) is invalid; when so, this is indicated at the start of the example. Feather Expires August 30, 2003 [Page 8] Internet-Draft Network News Transport Protocol March 2003 3. Basic Operation Every NNTP session MUST involve the following in this order: CONNECTION GREETING DISCONNECTION Other steps may occur between the GREETING and DISCONNECTION step. They are: CAPABILITIES DISCOVERY NEWS EXCHANGE CONCLUSION NNTP operates over any reliable data stream 8-bit-wide channel. When running over TCP/IP, the official port for the NNTP service is 119. Initially, the server host starts the NNTP service by listening on a TCP port. When a client host wishes to make use of the service, it MUST establish a TCP connection with the server host by connecting to that host on the same port on which the server is listening. This is the CONNECTION step. When the connection is established, the NNTP server host MUST send a greeting. This is the GREETING step. The client host and server host SHOULD then exchange commands and responses (respectively) until the connection is closed or aborted. This final step is called the DISCONNECTION step. If there is a CONCLUSION step, it MUST immediately precede the DISCONNECTION step. There MUST be only one CONNECTION, CONCLUSION and DISCONNECTION step for each NNTP session. All other steps MAY be repeated as needed. For example, the GREETING step may be repeated if the client makes use of the MODE READER command (see Section 5.2 for more on the MODE READER command). OUTSTANDING ISSUE Do we actually need this GREETING / NEWS EXCHANGE / DISCONNECTION type stuff? I don't see that it buys us anything compared with simply saying that there's the initial greeting and a set of commands. The character set for all NNTP commands is UTF-8. Commands in the NNTP MUST consist of a keyword, which MAY be followed by one or more arguments. An US-ASCII CRLF pair MUST terminate all commands. Multiple commands MUST NOT be on the same line. Keywords MUST consist of printable US-ASCII characters. Unless otherwise noted elsewhere in this document, arguments SHOULD consist of printable US-ASCII characters. Keywords and arguments MUST be each separated Feather Expires August 30, 2003 [Page 9] Internet-Draft Network News Transport Protocol March 2003 by one or more US-ASCII SPACE or US-ASCII TAB characters. Keywords MUST be at least three US-ASCII characters and MUST NOT exceed 12 US-ASCII characters. Command lines MUST NOT exceed 512 octets, which includes the terminating US-ASCII CRLF pair. The arguments MUST NOT exceed 497 octets. Commands may have variants, using a second keyword immediately after the first to indicate which variant is required. The only such commands in this specification are LIST and MODE. Keywords are case-insensitive; the case of keywords for commands MUST be ignored by the server. Command and response parameters are case or language specific only when specified (either in this document or in RFC 1036 [6]). An NNTP server MUST implement all the commands in this specification except for those marked as optional and those in extensions. Each response MUST start with a three-digit response code that is sufficient to distinguish all responses. Certain valid responses are defined to be multi-line; for all others, the response is contained in a single line. OUTSTANDING ISSUE Should the initial response line be limited to 512 octets as well? Possible text: The first or only line of the response MUST NOT exceed 512 octets, which includes the response code and the terminating US-ASCII CRLF pair. The text further down about "does not place any limit on the length" would need equivalent edits. All multi-line responses MUST adhere to the following format: 1. The response consists of a sequence of one or more "lines", each being a stream of octets ending with 0x0D 0x0A (US-ASCII CRLF). Apart from those line endings, the stream MUST NOT include the octets 0x00, 0x0A, or 0x0D (US-ASCII NUL, LF, and CR). 2. The first such line contains the response code as with a single line response. 3. If any subsequent line begins with the "termination octet" (0x2E or US_ASCII "."), that line MUST be "byte-stuffed" by pre-pending an additional termination octet (0x2E) to that line of the Feather Expires August 30, 2003 [Page 10] Internet-Draft Network News Transport Protocol March 2003 response. 4. The lines of the response MUST be followed by a terminating line consisting of a single termination octet (0x2E or US_ASCII ".") followed by CRLF in the normal way. Thus a multi-line response is always terminated with the five octets CRLF "." CRLF (in US-ASCII). 5. When interpreting a multi-line response, the "byte stuffing" MUST be undone; i.e. the client MUST ensure that, in any line beginning with the termination octet followed by octets other than US-ASCII CRLF, that initial termination octet is disregarded. 6. Likewise, the terminating line "." CRLF (in US-ASCII) MUST NOT be considered part of the multi-line response; i.e. the client MUST ensure that any line beginning with the termination octet followed immediately by US-ASCII CRLF is disregarded; (the first CRLF of the terminating CRLF "." CRLF is, of course, part of the last line of the response). Note that texts using an encoding (such as UTF-16 or UTF-32) that may contain the NUL octet or the CR or LF octets in contexts other than the CRLF line ending cannot be reliably conveyed in the above format. This document does not place any limit on the length of a line. However, the standards that define the format of articles may do so. An NNTP server MAY have an inactivity autologout timer. Such a timer SHOULD be of at least three minutes duration, with the exception that there MAY be a shorter limit on how long the server is willing to wait for the first command from the client. The receipt of any command from the client during the timer interval SHOULD suffice to reset the autologout timer. Similarly, the receipt of any significant amount of data from the client while in the midst of sending a multi-line message to the server (such as during a POST or IHAVE command) SHOULD suffice to reset the autologout timer. When the timer expires, the server SHOULD close the TCP connection without sending any response to the client, including when the client is in the middle of sending a multi-line message to the server. 3.1 Response Codes Each response MUST begin with a three-digit status indicator. These are status reports from the server and indicate the response to the last command received from the client. The first digit of the response broadly indicates the success, Feather Expires August 30, 2003 [Page 11] Internet-Draft Network News Transport Protocol March 2003 failure, or progress of the previous command. 1xx - Informative message. 2xx - Command completed OK. 3xx - Command OK so far; send the rest of it. 4xx - Command was correct, but couldn't be performed for some reason. 5xx - Command unimplemented, or incorrect, or a serious program error occurred. The next digit in the code indicates the function response category. x0x - Connection, setup, and miscellaneous messages x1x - Newsgroup selection x2x - Article selection x3x - Distribution functions x4x - Posting x8x - Reserved for authentication and authorization extensions x9x - Reserved for private use (non-standard extensions) Certain responses contain parameters such as numbers and names in addition to the status indicator. In those cases, to simplify interpretation by the client the number and type of such parameters is fixed for each response code, as is whether or not the code introduces a multi-line response. Any extension MUST follow this principle as well, but note that, for historical reasons, the 211 response code is an exception to this. In all other cases, the client MUST only use the status indicator itself to determine the nature of the response. The exact response codes that can be returned by any given command are detailed in the description of that command. Parameters MUST be separated from the numeric status indicator and from each other by a single US-ASCII space. All numeric parameters MUST be in base 10 (decimal) format, and MAY have leading zeros. String parameters MUST contain at least one character and MUST NOT contain US-ASCII spaces, CR, LF, or tab. The server MAY add any text after the response code or last parameter as appropriate, and the client MUST NOT make decisions based on this text. Such text MUST be separated from the numeric status indicator or the last parameter by at least one US-ASCII space. The server MUST respond to any command with the appropriate generic response (given in Section 3.1.1) if it represents the situation. Otherwise, each recognized command MUST return one of the response codes specifically listed in its description or in an extension. A server MAY provide extensions to this specification, including new commands, new variants or features of existing commands, and other Feather Expires August 30, 2003 [Page 12] Internet-Draft Network News Transport Protocol March 2003 ways of changing the internal state of the server. However, the server MUST NOT produce any other responses to a client that does not invoke any of the additional features. (Therefore a client that restricts itself to this specification will only receive the responses that are listed.) If a client receives an unexpected response, it SHOULD use the first digit of the response to determine the result. For example, an unexpected 2xx should be taken as success and an unexpected 4xx or 5xx as failure. Response codes not specified in this document MAY be used for any installation-specific additional commands also not specified. These SHOULD be chosen to fit the pattern of x9x specified above. Neither this document nor any extension registered with IANA (see Section 10) will specify any response codes of the x9x pattern. (Implementers of extensions are accordingly cautioned not to use such responses for extensions that may subsequently be submitted for registration.) 3.1.1 Generic Response Codes The server MUST respond to any command with the appropriate one of the following generic responses if it represents the situation. If the command is not recognized, or it is an optional command or extension that is not implemented by the server, the response code 500 MUST be returned. If there is a syntax error in the arguments of a recognized command, including the case where more arguments are provided than the command specifies, the response code 501 MUST be returned. Note that where a command has variants depending on a second keyword (e.g. LIST ACTIVE and LIST NEWSGROUPS), then 501 MUST be used when the requested variant is not implemented but the base command is. If the client is not authorized to use the specified facility when the server is in its current state, the response code 502 MUST be returned. A different command might change the server state and permit the command if it is retried. If the server does not provide an optional feature, then the response code 403 MUST be returned if the omission is temporary (e.g. because a necessary facility is unavailable) and the code 503 if it is permanent (e.g. because the server does not store the required information). Feather Expires August 30, 2003 [Page 13] Internet-Draft Network News Transport Protocol March 2003 OUTSTANDING ISSUE Is anyone aware of a server that implements 403, or is it an invention of our own? If the latter, do we want to keep it? INN apparently uses 503 for temporary errors; someone suggested adding the text: If the server encounters an unexpected internal error that prevents it from completing a command, the response code 503 MAY be returned. Some servers return 503 for things like "can't contact a posting server" or "can't execute external authenticator". OUTSTANDING ISSUE The 503 response seems to have three separate meanings: 1. LIST ACTIVE.TIMES etc. use it for "this data isn't stored". HDR uses it for "this header can't be requested", which is consistent. Are there other commands that can reasonably return such a thing? If not, is this kind of 503 really a generic response? 2. Temporary errors, the kind that 403 is supposed to represent. 3. It's apparently returned by LIST EXTENSIONS, but what does it mean in this case? Not "there are no extensions", because that's 402. Is this also an invention of our own? Again, would a different code be better? If the server has to terminate the connection for some reason, it MUST give a 400 response code to the next command and then immediately close the TCP connection. It MAY give a 401 response code to any command to indicate that termination is imminent (following a 401 response, it MUST NOT close the TCP connection immediately). OUTSTANDING ISSUE Since the 401 doesn't terminate the session, what about commands that change the status? For example, if GROUP returns 401 what happens to the current selected newsgroup. With the exception of mandatory commands and the 500 response, the client MUST be prepared to receive any of these responses for any command. Feather Expires August 30, 2003 [Page 14] Internet-Draft Network News Transport Protocol March 2003 3.1.1.1 Examples Example of an unknown command: [C] MAIL [S] 500 Unknown command Example of an unsupported extension: [C] LIST EXTENSIONS [S] 202 Extensions supported: [S] LISTGROUP [S] . [C] OVER [S] 500 Unknown command Example of an unsupported variant: [C] MODE POSTER [S] 501 Unknown MODE option Example of a syntax error: [C] ARTICLE a.message.id@no.angle.brackets [S] 501 Syntax error Example of an overlong command line: [C] HEAD 53 54 55 [S] 501 Too many arguments Example of a bad wildmat: [C] LIST ACTIVE u[ks].* [S] 501 Syntax error Example of an attempt to access a restricted facility: [C] GROUP secret.group [S] 502 Permission denied followed by a successful attempt following authentication: [C] XSECRET fred flintstone [S] 290 Password for fred accepted. [C] GROUP secret.group [S] 211 5 1 20 secret.group selected Feather Expires August 30, 2003 [Page 15] Internet-Draft Network News Transport Protocol March 2003 Example of a temporary failure: [C] GROUP archive.local [S] 403 Archive server temporarily offline Example of the server needing to close down immediately: [C] ARTICLE 123 [S] 400 Power supply failed, running on UPS [Server closes connection.] Example of imminent termination of the server: [C] STAT 123 [S] 401 Pre-payment expired, you have 10 seconds [C] STAT 123 [S] 423 No such article number in this group [C] NEXT [S] 400 Time expired [Server closes connection.] 3.2 Pipelining NNTP is designed to operate over a reliable bi-directional connection such as TCP. Therefore, if a command does not depend on the response to the previous one, it should not matter if it is sent before that response is received. Doing this is called "pipelining". However, certain server implementations throw away all text received from the client following certain commands before sending their response. If this happens, pipelining will be affected because one or more commands will have been ignored or misinterpreted, and the client will be matching the wrong responses to each command. Since there are significant benefits to pipelining, but also circumstances where it is reasonable or common for servers to behave in the above manner, this document puts certain requirements on both clients and servers. Except where stated otherwise, a client MAY use pipelining. That is, it may send a command before receiving the response for the previous command. The server MUST allow pipelining and MUST NOT throw away any text received after a command. Irrespective of whether or not pipelining is used, the server MUST process commands in the order they are sent. If the specific description of a command say it "MUST NOT be pipelined", that command MUST end any pipeline of commands. That is, the client MUST NOT send any following command until receiving the CRLF at the end of the response from the command. The server MAY Feather Expires August 30, 2003 [Page 16] Internet-Draft Network News Transport Protocol March 2003 ignore any data received after the command and before the CRLF at the end of the response is sent to the client. The initial connection must not be part of a pipeline; that is, the client MUST NOT send any command until receiving the CRLF at the end of the greeting. If the client uses blocking system calls to send commands, it MUST ensure that the amount of text sent in pipelining does not cause a deadlock between transmission and reception. The amount of text involved will depend on window sizes in the transmission layer, and is typically 4k octets for TCP. 3.2.1 Examples Example of correct use of pipelining: [C] GROUP misc.test [C] STAT [C] NEXT [S] 211 1234 3000234 3002322 misc.test [S] 223 3000234 <45223423@example.com> retrieved [S] 223 3000237 <668929@example.org> retrieved Example of incorrect use of pipelining (the LIST EXTENSIONS command may not be pipelined): [C] GROUP misc.test [C] LIST EXTENSIONS [C] DATE [C] NEXT [S] 211 1234 3000234 3002322 misc.test [S] 402 server has no extensions [S] 223 3000237 <668929@example.org> retrieved The DATE command has been thrown away by the server and so there is no 111 response to match it. Feather Expires August 30, 2003 [Page 17] Internet-Draft Network News Transport Protocol March 2003 4. The WILDMAT format The WILDMAT format described here is based on the version first developed by Rich Salz [11], which in turn was derived from the format used in the UNIX "find" command to articulate file names. It was developed to provide a uniform mechanism for matching patterns in the same manner that the UNIX shell matches filenames. 4.1 Wildmat syntax A wildmat is described by the following augmented BNF [5] syntax (note that this syntax contains ambiguities and special cases described at the end): wildmat = wildmat-pattern *("," ["!"] wildmat-pattern) wildmat-pattern = 1*wildmat-item wildmat-item = wildmat-exact / wildmat-wild wildmat-exact = %x21-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E / UTF-8-non-ascii ; exclude * , ? [ \ ] wildmat-wild = "*" / "?" UTF-8-non-ascii is defined in Section 11 This syntax must be interpreted subject to the following rule: Where a wildmat-pattern is not immediately preceded by "!", it shall not begin with a "!". Note: the characters \ , [ and ] are not allowed in wildmats, while * and ? are always wildcards. This should not be a problem since these characters cannot occur in newsgroup names, which is the only current use of wildmats. Backslash is commonly used to supress the special meaning of characters and brackets to introduce sets, but there is no existing standard practice for these in wildmats and so they were omitted from this specification. A future extension to this specification may provide semantics for these characters. 4.2 Wildmat semantics A wildmat is tested against a string, and either matches or does not match. To do this, each constituent wildmat-pattern is matched against the string and the rightmost pattern that matches is identified. If that wildmat-pattern is not preceded with "!", the whole wildmat matches. If it is preceded by "!", or if no Feather Expires August 30, 2003 [Page 18] Internet-Draft Network News Transport Protocol March 2003 wildmat-pattern matches, the whole wildmat does not match. For example, consider the wildmat "a*,!*b,*c*": the string "aaa" matches because the rightmost match is with "a*" the string "abb" does not match because the rightmost match is with "*b" the string "ccb" matches because the rightmost match is with "*c*" the string "xxx" does not match because no wildmat-pattern matches A wildmat-pattern matches a string if the string can be broken into components, each of which matches the corresponding wildmat-item in the pattern; the matches must be in the same order, and the whole string must be used in the match. The pattern is "anchored"; that is, the first and last characters in the string must match the first and last item respectively (unless that item is an asterisk matching zero characters). A wildmat-exact matches the same character (which may be more than one octet in UTF-8). "?" matches exactly one character (which may be more than one octet). "*" matches zero or more characters. It can match an empty string, but it cannot match a subsequence of a UTF-8 sequence that is not aligned to the character boundaries. 4.3 Extensions An NNTP server or extension MAY extend the syntax or semantics of wildmats provided that all wildmats that meet the requirements of Section 4.1 have the meaning ascribed to them by Section 4.2. Future editions of this document may also extend wildmats. Feather Expires August 30, 2003 [Page 19] Internet-Draft Network News Transport Protocol March 2003 4.4 Examples In these examples, $ and @ are used to represent the two octets 0xC2 and 0xA3 respectively; $@ is thus the UTF-8 encoding for the pound sterling symbol, shown as # in the descriptions. Wildmat Description of strings that match abc the one string "abc" abc,def the two strings "abc" and "def" $@ the one character string "#" a* any string that begins with "a" a*b any string that begins with "a" and ends with "b" a*,*b any string that begins with "a" or ends with "b" a*,!*b any string that begins with "a" and does not end with "b" a*,!*b,c* any string that begins with "a" and does not end with "b", and any string that begins with "c" no matter what it ends with a*,c*,!*b any string that begins with "a" or "c" and does not end with "b" ?a* any string with "a" as its second character ??a* any string with "a" as its third character *a? any string with "a" as its penultimate character *a?? any string with "a" as its antepenultimate character Feather Expires August 30, 2003 [Page 20] Internet-Draft Network News Transport Protocol March 2003 5. The GREETING Step 5.1 Initial Connection 5.1.1 Usage Responses 200 Service available, posting allowed 201 Service available, posting prohibited 400 Service temporarily unavailable [1] 502 Service permanently unavailable [1] These are the only valid response codes for the initial greeting; the server MUST not return any other generic response code. [1] Following a 400 or 502 response the server MUST immediately close the connection. 5.1.2 Description There is no command presented by the client upon initial connection to the server. The server MUST present an appropriate response code as a greeting to the client. This response informs the client about what steps the client should take to reach the news exchange step. If the server will accept further commands from the client including POST, the server MUST present a 200 greeting code. If the server will accept further commands from the client, but it is not authorized to post articles using the POST command, the server MUST present a 201 greeting code. Otherwise the server MUST present a 400 or 502 greeting code and then immediately close the connection. 502 MUST be used if the client is not permitted under any circumstances to interact with the server and 400 otherwise. 5.1.3 Examples Example of a normal connection from an authorized client which then jumps directly to the conclusion step (see Section 9): [Initial TCP connection setup completed.] [S] 200 NNTP Service Ready, posting permitted [C] QUIT [S] 205 NNTP Service exits normally [Server closes connection.] Feather Expires August 30, 2003 [Page 21] Internet-Draft Network News Transport Protocol March 2003 Example of a normal connection from an authorized client that is not permitted to post; it also jumps directly to the conclusion step: [Initial TCP connection setup completed.] [S] 201 NNTP Service Ready, posting prohibited [C] QUIT [S] 205 NNTP Service exits normally [Server closes connection.] Example of a normal connection from an unauthorized client: [Initial TCP connection setup completed.] [S] 502 NNTP Service permanently unavailable [Server closes connection.] Example of a connection from a client where the server is unable to provide service: [Initial TCP connection setup completed.] [S] 400 NNTP Service temporarily unavailable [Server closes connection.] 5.2 MODE READER 5.2.1 Usage This command MUST NOT be pipelined. Syntax MODE READER Responses 200 Posting allowed 201 Posting prohibited 400 Service temporarily unavailable [1] 502 Service permanently unavailable [1] [1] Following a 400 or 502 response the server MUST immediately close the connection. 5.2.2 Description MODE READER SHOULD be sent by any client that intends to use any command other than IHAVE, HEAD, STAT, LIST ACTIVE, LIST EXTENSIONS, or commands advertised by the server as available via LIST EXTENSIONS. Feather Expires August 30, 2003 [Page 22] Internet-Draft Network News Transport Protocol March 2003 Servers MAY require that this command be issued before any other commands are sent and MAY reject any other commands until after a MODE READER command has been sent. The server MUST return a response using the same codes as the initial greeting (as described in Section 5.1.1) to indicate its ability to provide reading service to the client. Note that the response need not be the same as the one presented during the initial greeting. Once MODE READER is sent, IHAVE (and any extensions intended for peer-to-peer article transfer) MAY no longer be permitted, even if it were permitted before the MODE READER command. The results of LIST EXTENSIONS MAY be different following a MODE READER command than prior to the issuing of that command. Servers are encouraged to not require this command even though clients SHOULD send it when appropriate. It is present to support some news architectures that switch between modes based on whether a given connection is a peer-to-peer connection with another server or a news reading client. 5.2.3 Examples Example of use of the MODE READER command by an authorized client which then jumps directly to the conclusion step (see Section 9): [C] MODE READER [S] 200 NNTP Service Ready, posting permitted [C] QUIT [S] 205 NNTP Service exits normally [Server closes connection.] Example of use of the MODE READER command by an authorized client that is not permitted to post; it also jumps directly to the conclusion step: [C] MODE READER [S] 201 NNTP Service Ready, posting prohibited [C] QUIT [S] 205 NNTP Service exits normally [Server closes connection.] Example of use of MODE READER by a client not authorized to receive service from the server as a news reader: [C] MODE READER [S] 502 NNTP Service permanently unavailable [Server closes connection.] Feather Expires August 30, 2003 [Page 23] Internet-Draft Network News Transport Protocol March 2003 Example of a connection from any client where the server is unable to provide news reader service: [C] QUIT [S] 400 NNTP Service temporarily unavailable [Server closes connection.] Feather Expires August 30, 2003 [Page 24] Internet-Draft Network News Transport Protocol March 2003 6. The CAPABILITIES DISCOVERY step To discover what extensions are available, an NNTP client can query the server with the LIST EXTENSIONS command. If a particular extension is unavailable, the client can attempt to work around it or it may wish to terminate the session. See Section 10 for further discussion of extensions. 6.1 LIST EXTENSIONS 6.1.1 Usage This command is optional. This command MUST NOT be pipelined. Syntax LIST EXTENSIONS Responses 202 Extension list follows (multiline) 402 Server has no extensions 503 Extension information not available 6.1.2 Description The LIST EXTENSIONS command allows a client to determine which extensions are supported by the server. This command MUST be implemented by any server that implements any extensions defined in this document. To discover what extensions are available, an NNTP client SHOULD query the server early in the session for extensions information by issuing the LIST EXTENSIONS command. This command MAY be issued at anytime during a session. It is not required that the client issues this command before attempting to make use of any extension. The response generated by this command MAY change during a session because of other state information. However, an NNTP client MUST NOT cache (for use in another session) any information returned if the LIST EXTENSIONS command succeeds. That is, an NNTP client is only able to get the current and correct information concerning available extensions during a session by issuing a LIST EXTENSIONS command during that session and processing that response. The list of extensions is returned as a multi-line response following the 202 response code. Each extension is listed on a separate line; Feather Expires August 30, 2003 [Page 25] Internet-Draft Network News Transport Protocol March 2003 the line MUST begin with an extension-label and optionally one or more parameters (separated by single spaces). The extension-label and the meaning of the parameters are specified as part of the definition of the extension. The extension-label MUST be in uppercase. The server MUST NOT list the same extension twice in the response, and MUST list all supported extensions. The order in which the extensions are listed is not significant. The server need not even consistently return the same order. If the server does not support any extensions, a 402 response SHOULD be returned, but it MAY instead return an empty list. Following a 503 response an extension might still be available, and the client MAY attempt to use it. 6.1.3 Examples Example of a successful response: [C] LIST EXTENSIONS [S] 202 Extensions supported: [S] OVER [S] HDR [S] LISTGROUP [S] . The particular extensions shown here are simply examples of what might be defined in other places, and no particular meaning should be attributed to them. Example where no extensions are available, using preferred format: [C] LIST EXTENSIONS [S] 402 Server has no extensions Example where no extensions are available, using an empty list: [C] LIST EXTENSIONS [S] 202 Extensions supported: [S] . Feather Expires August 30, 2003 [Page 26] Internet-Draft Network News Transport Protocol March 2003 7. Article posting and retrieval News reading clients have available a variety of mechanisms to retrieve articles via NNTP. The news articles are stored and indexed using three types of keys. One key is the message-id of an article. According to RFC 1036, this identifier should be globally unique. Another key is composed of the newsgroup name and the article number within that newsgroup. That key MUST be unique to a particular server (there will be only one article with that number within a particular newsgroup), but is not required to be globally unique. Additionally, because the same article can be cross-posted to multiple newsgroups, there may be multiple keys that point to the same article on the same server. The final key is the arrival timestamp, giving the time that the article arrived at the server. The server MUST ensure that article numbers are issued in order of arrival timestamp; that is, articles arriving later MUST have higher numbers than those that arrive earlier. The server SHOULD allocate the next sequential unused number to each new article. Article numbers MUST lie between 1 and 4,294,967,295 inclusive. The client and server SHOULD NOT use leading zeroes in specifying article numbers, and MUST NOT use more than 16 digits. In some situations, the value zero replaces an article number to show some special situation. Message-ids are as defined in RFC 2822 [7] with the following modifications: o A message-id MUST NOT contain a US-ASCII space within any quoted-pair. o A message-id MUST NOT be longer than 250 octets. o RFC 2822 obsolete syntax for message-ids is not supported by the protocol specified in this document. 7.1 Group and article selection The following commands are used to set the "current selected newsgroup" and the "current article number", which are used by various commands. At the start of an NNTP session, both of these values are set to the special value "invalid". 7.1.1 GROUP Feather Expires August 30, 2003 [Page 27] Internet-Draft Network News Transport Protocol March 2003 7.1.1.1 Usage Syntax GROUP ggg Responses 211 n l h ggg Group successfully selected 411 No such newsgroup Parameters ggg = name of newsgroup n = estimated number of articles in the group l = reported low water mark h = reported high water mark 7.1.1.2 Description The required parameter ggg is the name of the newsgroup to be selected (e.g. "news.software.b"). A list of valid newsgroups may be obtained by using the LIST ACTIVE command (see Section 8.6.1). The successful selection response will return the article numbers of the first and last articles in the group at the moment of selection (these numbers are referred to as the "reported low water mark" and the "reported high water mark"), and an estimate of the number of articles on file in the group. If the group is not empty, the estimate MUST be at least the actual number of articles available, and MUST be no greater than one more than the difference between the reported low and high water marks. (Some implementations will actually count the number of articles on file. Others will just subtract the low water mark from the high water mark and add one to get an estimate.) If the group is empty, one of the following three situations will occur. Clients MUST accept all three cases; servers MUST NOT represent an empty group in any other way. o The high water mark will be one less than the low water mark, and the estimated article count will be zero. Servers SHOULD use this method to show an empty group. This is the only time that the high water mark can be less than the low water mark. o All three numbers will be zero. o The high water mark is greater than or equal to the low water mark. The estimated article count might be zero or non-zero; if Feather Expires August 30, 2003 [Page 28] Internet-Draft Network News Transport Protocol March 2003 non-zero, the same requirements apply as for a non-empty group. The set of articles in a group may change after the GROUP command is carried out. That is: o articles may be removed from the group o articles may be reinstated in the group with the same article number, but those articles MUST have numbers no less than the reported low water mark (note that this is a reinstatement of the previous article, not a new article reusing the number) o new articles may be added with article numbers greater than the reported high water mark (if an article that was the one with the highest number has been removed, the next new article will not have the number one greater than the reported high water mark) Except when the group is empty and all three numbers are zero, whenever a subsequent GROUP command for the same newsgroup is issued, either by the same client or a different client, the reported low water mark in the response MUST be no less than that in any previous response for that newsgroup sent to any client. The client may make use of the low water mark to remove all remembered information about articles with lower numbers, as these will never recur. This includes the situation when the high water mark is one less than the low water mark. No similar assumption can be made about the high water mark, as this can decrease if an article is removed, and then increase again if it is reinstated or if new articles arrive. When a valid group is selected by means of this command, the current selected newsgroup MUST be set to that group and the current article number MUST be set to the first article in the group. If an empty newsgroup is selected, the current article pointer is made invalid. If an invalid group is specified, the current selected newsgroup and current article number MUST NOT be changed. The GROUP command (or the LISTGROUP command, if implemented) MUST be used by a client and a successful response received before the any other command is used that depends on the value of the current selected newsgroup or current article number. If the group specified is not available on the server, a 411 response MUST be returned. 7.1.1.3 Examples Example for a group known to the server: Feather Expires August 30, 2003 [Page 29] Internet-Draft Network News Transport Protocol March 2003 [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test Example for a group unknown to the server: [C] GROUP example.is.sob.bradner.or.barber [S] 411 example.is.sob.bradner.or.barber is unknown Example of an empty group using the preferred response: [C] GROUP example.currently.empty.newsgroup [S] 211 0 4000 3999 example.currently.empty.newsgroup Example of an empty group using an alternative response: [C] GROUP example.currently.empty.newsgroup [S] 211 0 0 0 example.currently.empty.newsgroup Example of an empty group using a different alternative response: [C] GROUP example.currently.empty.newsgroup [S] 211 0 4000 4321 example.currently.empty.newsgroup 7.1.2 LAST 7.1.2.1 Usage Syntax LAST Responses 223 n message-id Article found 412 No newsgroup selected 420 Current article number is invalid 422 No previous article in this group Parameters n = article number message-id = article message-id 7.1.2.2 Description If the current selected newsgroup is valid, the current article number MUST be set to the previous article in that newsgroup (that is, the highest existing article number less than the current article number). If successful, a response indicating the new current Feather Expires August 30, 2003 [Page 30] Internet-Draft Network News Transport Protocol March 2003 article number and the message-id of that article MUST be returned. No article text is sent in response to this command. There MAY be no previous article in the group, although the current article number is not the reported low water mark. There MUST NOT be a previous article when the current article number is the reported low water mark. Because articles can be removed and added, the results of multiple LAST and NEXT commands MAY not be consistent over the life of a particular NNTP session. If the current article number is already the first article of the newsgroup, a 422 response MUST be returned. If the current article number is invalid, a 420 response MUST be returned. If the current selected newsgroup is invalid, a 412 response MUST be returned. In all three cases the current selected newsgroup and current article number MUST NOT be altered. 7.1.2.3 Examples Example of a successful article retrieval using LAST: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] NEXT [S] 223 3000237 <668929@example.org> retrieved [C] LAST [S] 223 3000234 <45223423@example.com> retrieved Example of an attempt to retrieve an article without having selected a group (via the GROUP command) first: [Assumes current selected newsgroup is invalid.] [C] LAST [S] 412 no newsgroup selected Example of an attempt to retrieve an article using the LAST command when the current article number is that of the first article in the group: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] LAST [S] 422 No previous article to retrieve Example of an attempt to retrieve an article using the LAST command when the current selected newsgroup is empty: Feather Expires August 30, 2003 [Page 31] Internet-Draft Network News Transport Protocol March 2003 [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] LAST [S] 420 No current article selected 7.1.3 NEXT 7.1.3.1 Usage Syntax NEXT Responses 223 n message-id Article found 412 No newsgroup selected 420 Current article number is invalid 421 No next article in this group Parameters n = article number message-id = article message-id 7.1.3.2 Description If the current selected newsgroup is valid, the current article number MUST be set to the next article in that newsgroup (that is, the lowest existing article number greater than the current article number). If successful, a response indicating the new current article number and the message-id of that article MUST be returned. No article text is sent in response to this command. If the current article number is already the last article of the newsgroup, a 421 response MUST be returned. In all other aspects (apart, of course, from the lack of 422 response) this command is identical to the LAST command (Section 7.1.2). 7.1.3.3 Examples Example of a successful article retrieval using NEXT: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] NEXT [S] 223 3000237 <668929@example.org> retrieved Example of an attempt to retrieve an article without having selected Feather Expires August 30, 2003 [Page 32] Internet-Draft Network News Transport Protocol March 2003 a group (via the GROUP command) first: [Assumes current selected newsgroup is invalid.] [C] NEXT [S] 412 no newsgroup selected Example of an attempt to retrieve an article using the NEXT command when the current article number is that of the last article in the group: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] STAT 3002322 [S] 223 3002322 <411@example.net> retrieved [C] NEXT [S] 421 No next article to retrieve Example of an attempt to retrieve an article using the NEXT command when the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] NEXT [S] 420 No current article selected 7.2 Retrieval of articles and article sections The ARTICLE, BODY, HEAD, and STAT commands are very similar. They differ only in the parts of the article that are presented to the client and in the successful response code. The ARTICLE command is described here in full, while the other commands are described in terms of the differences. An article, as defined by RFC 1036, consists of two parts: the article headers and the article body. When responding to one of these commands, the server presents the entire article or appropriate part and does not attempt to alter or translate it in any way. 7.2.1 ARTICLE 7.2.1.1 Usage Syntax ARTICLE message-id ARTICLE [number] Feather Expires August 30, 2003 [Page 33] Internet-Draft Network News Transport Protocol March 2003 Responses First form (message-id specified) 220 0 message-id Article follows (multiline) 430 No article found with that message-id Second form (optional article number specified) 220 n message-id Article follows (multiline) 412 No newsgroup selected 420 Current article number is invalid [1] 423 No such article in this newsgroup Parameters number = Requested article number n = Returned article number message-id = Article message-id [1] The 420 response can only occur if no article number has been specified. 7.2.1.2 Description The ARTICLE command selects an article based on the arguments and presents the header, a blank line, and the body of that article. The command has two forms. In the first form, a message-id is specified (including the angle brackets), and the server presents the article with that message-id in its headers. In this case, the server MUST NOT alter the current selected newsgroup or current article number. This is both to facilitate the presentation of articles that may be referenced within another article being read, and because of the semantic difficulties of determining the proper sequence and membership of an article that may have been crossposted to more than one newsgroup. In the response, the article number is replaced with zero (that is, the server is not required to determine whether the article is in the current group or what article number(s) it has). In the second form, an article number may be specified. If so, and if there is an article with that number in the currently selected newsgroup, the server MUST set the current article number to that number. Then, whether or not a number was specified, the article indicated by the current article number is presented to the client. Feather Expires August 30, 2003 [Page 34] Internet-Draft Network News Transport Protocol March 2003 Note that a previously valid article number MAY become invalid if the article has been removed. A previously invalid article number MAY become valid if the article has been reinstated, but such an article number MUST be no less than the reported low water mark for that group. The server MUST NOT change the current selected newsgroup as a result of this command. The server MUST NOT change the current article number except when an article number argument was provided and the article exists; in particular, it MUST NOT change it following an unsuccessful response. The message-id of the article is taken from the message-id header line of the article (required by RFC 1036). If there is no such line, the message-id "<0>" MUST be used instead (without the double quotes). Since the message-id field is unique for each article, it may be used by a client to skip duplicate displays of articles that have been posted more than once, or to more than one newsgroup. The article headers and body are returned as a multi-line response following the 220 response code. If the current article number is invalid, a 420 response MUST be returned. If there is no article with the specified number, a 423 response MUST be returned. If the current selected newsgroup is invalid, a 412 response MUST be returned. 7.2.1.3 Examples Example of a successful retrieval of an article (using no article number): [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] ARTICLE [S] 220 3000234 <45223423@example.com> [S] Path: pathost!demo!whitehouse!not-for-mail [S] From: "Demo User" [S] Newsgroups: misc.test [S] Subject: I am just a test article [S] Date: 6 Oct 1998 04:38:40 -0500 [S] Organization: An Example Net, Uncertain, Texas [S] Message-ID: <411@example.net> [S] [S] This is just a test article. [S] . Feather Expires August 30, 2003 [Page 35] Internet-Draft Network News Transport Protocol March 2003 Example of a successful retrieval of an article by message-id: [C] ARTICLE <45223423@example.com> [S] 220 0 <45223423@example.com> [S] Path: pathost!demo!whitehouse!not-for-mail [S] From: "Demo User" [S] Newsgroups: misc.test [S] Subject: I am just a test article [S] Date: 6 Oct 1998 04:38:40 -0500 [S] Organization: An Example Net, Uncertain, Texas [S] Message-ID: <411@example.net> [S] [S] This is just a test article. [S] . Example of an unsuccessful retrieval of an article by message-id: [C] ARTICLE [S] 430 No Such Article Found Example of an unsuccessful retrieval of an article by number: [C] GROUP misc.test [S] 211 1234 3000234 3002322 news.groups [C] ARTICLE 300256 [S] 423 No such article number in this group Example of an unsuccessful retrieval of an article by number because no newsgroup was selected first: [Assumes current selected newsgroup is invalid.] [C] ARTICLE 300256 [S] 412 No newsgroup selected Example of an attempt to retrieve an article when the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] ARTICLE [S] 420 No current article selected 7.2.2 HEAD 7.2.2.1 Usage Feather Expires August 30, 2003 [Page 36] Internet-Draft Network News Transport Protocol March 2003 Syntax HEAD message-id HEAD [number] Responses First form (message-id specified) 221 0 message-id Headers follow (multiline) 430 No article found with that message-id Second form (optional article number specified) 221 n message-id Headers follow (multiline) 412 No newsgroup selected 420 Current article number is invalid [1] 423 No such article in this newsgroup Parameters number = Requested article number n = Returned article number message-id = Article message-id [1] The 420 response can only occur if no article number has been specified. 7.2.2.2 Description The HEAD command behaves identically to the ARTICLE command except that, if the article exists, the response code is 221 instead of 220 and only the headers are presented (the blank line separating the headers and body MUST NOT be included). 7.2.2.3 Examples Example of a successful retrieval of the headers in an article (using no article number): [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] HEAD [S] 221 3000234 <45223423@example.com> [S] Path: pathost!demo!whitehouse!not-for-mail [S] From: "Demo User" [S] Newsgroups: misc.test [S] Subject: I am just a test article [S] Date: 6 Oct 1998 04:38:40 -0500 [S] Organization: An Example Net, Uncertain, Texas [S] Message-ID: <411@example.net> Feather Expires August 30, 2003 [Page 37] Internet-Draft Network News Transport Protocol March 2003 [S] . Example of a successful retrieval of the headers in an article by message-id: [C] HEAD <45223423@example.com> [S] 221 0 <45223423@example.com> [S] Path: pathost!demo!whitehouse!not-for-mail [S] From: "Demo User" [S] Newsgroups: misc.test [S] Subject: I am just a test article [S] Date: 6 Oct 1998 04:38:40 -0500 [S] Organization: An Example Net, Uncertain, Texas [S] Message-ID: <411@example.net> [S] . Example of an unsuccessful retrieval of the header of an article by message-id: [C] HEAD [S] 430 No Such Article Found Example of an unsuccessful retrieval of the header of an article by number: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] HEAD 300256 [S] 423 No such article number in this group Example of an unsuccessful retrieval the header of an article by number because no newsgroup was selected first: [Assumes current selected newsgroup is invalid.] [C] HEAD 300256 [S] 412 No newsgroup selected Example of an attempt to retrieve the header of an article when the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] HEAD [S] 420 No current article selected Feather Expires August 30, 2003 [Page 38] Internet-Draft Network News Transport Protocol March 2003 7.2.3 BODY 7.2.3.1 Usage Syntax BODY message-id BODY [number] Responses First form (message-id specified) 222 0 message-id Body follows (multiline) 430 No article found with that message-id Second form (optional article number specified) 222 n message-id Body follows (multiline) 412 No newsgroup selected 420 Current article number is invalid [1] 423 No such article in this newsgroup Parameters number = Requested article number n = Returned article number message-id = Article message-id [1] The 420 response can only occur if no article number has been specified. 7.2.3.2 Description The BODY command behaves identically to the ARTICLE command except that, if the article exists, the response code is 222 instead of 220 and only the body is presented (the blank line separating the headers and body MUST NOT be included). 7.2.3.3 Examples Example of a successful retrieval of the body of an article (using no article number): [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] BODY [S] 222 3000234 <45223423@example.com> [S] This is just a test article. [S] . Feather Expires August 30, 2003 [Page 39] Internet-Draft Network News Transport Protocol March 2003 Example of a successful retrieval of the body of an article by message-id: [C] BODY <45223423@example.com> [S] 222 0 <45223423@example.com> [S] This is just a test article. [S] . Example of an unsuccessful retrieval of the body of an article by message-id: [C] BODY [S] 430 No Such Article Found Example of an unsuccessful retrieval of the body of an article by number: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] BODY 300256 [S] 423 No such article number in this group Example of an unsuccessful retrieval of the body of an article by number because no newsgroup was selected first: [Assumes current selected newsgroup is invalid.] [C] BODY 300256 [S] 412 No newsgroup selected Example of an attempt to retrieve the body of an article when the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] BODY [S] 420 No current article selected 7.2.4 STAT 7.2.4.1 Usage Syntax STAT message-id STAT [number] Feather Expires August 30, 2003 [Page 40] Internet-Draft Network News Transport Protocol March 2003 Responses First form (message-id specified) 223 0 message-id Article exists 430 No article found with that message-id Second form (optional article number specified) 223 n message-id Article exists 412 No newsgroup selected 420 Current article number is invalid [1] 423 No such article in this newsgroup Parameters number = Requested article number n = Returned article number message-id = Article message-id [1] The 420 response can only occur if no article number has been specified. 7.2.4.2 Description The STAT command behaves identically to the ARTICLE command except that, if the article exists, it is NOT presented to the client and the response code is 223 instead of 220. Note that the response is NOT multi-line. This command allows the client to determine whether an article exists, and in the second form what its message-id is, without having to process an arbitrary amount of text. 7.2.4.3 Examples Example of STAT on an existing article (using no article number): [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] STAT [S] 223 3000234 <45223423@example.com> Example of a STAT of an existing article by message-id: [C] STAT <45223423@example.com> [S] 223 0 <45223423@example.com> Example of an STAT of an article not on the server by message-id: Feather Expires August 30, 2003 [Page 41] Internet-Draft Network News Transport Protocol March 2003 [C] STAT [S] 430 No Such Article Found Example of STAT of an article not in the server by number: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] STAT 300256 [S] 423 No such article number in this group Example of STAT of an article by number when no newsgroup was selected first: [Assumes current selected newsgroup is invalid.] [C] STAT 300256 [S] 412 No newsgroup selected Example of STAT of an article when the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] STAT [S] 420 No current article selected 7.3 Article posting Article posting is done in one of two modes: individual article posting from news reading clients using POST, and article transfer from other news servers using IHAVE. 7.3.1 POST 7.3.1.1 Usage This command MUST NOT be pipelined. Syntax POST Responses Initial responses 340 Send article to be posted 440 Posting not permitted Feather Expires August 30, 2003 [Page 42] Internet-Draft Network News Transport Protocol March 2003 Subsequent responses 240 Article received OK 441 Posting failed 7.3.1.2 Description If posting is allowed, a 340 response MUST be returned to indicate that the article to be posted should be sent. If posting is prohibited for some installation-dependent reason, a 440 response MUST be returned. If posting is permitted, the article MUST be presented to the server by the client in the format specified by RFC 1036 (or by any of its successors or extensions). The text forming the header and body of the message to be posted MUST be sent by the client in the format defined above (Section 3) for multi-line responses (except that there is no initial line containing a response code). Thus a single dot (".") on a line indicates the end of the text, and lines starting with a dot in the original text have that dot doubled during transmission. Following the presentation of the termination sequence by the client, the server MUST return a response indicating success or failure of the article transfer. Note that response codes 340 and 440 are used in direct response to the POST command. Others are returned following the sending of the article. A response of 240 SHOULD indicate that, barring unforseen server errors, the posted article will be made available on the server and/ or transferred to other servers as appropriate. In other words, articles not wanted by the server SHOULD be rejected with a 411 response and not accepted and silently discarded. No attempt shall be made by the server to filter characters, fold or limit lines, or otherwise process incoming text. The intent is that the server just passes the incoming message to be posted to the server installation's news posting software, which is not defined by this document. The client SHOULD NOT assume that the article has been successfully transferred unless it receives an affirmative response from the server. If the session is interrupted before the response is received, it is possible that an affirmative response was sent but has been lost. Therefore, in any subsequent session the client SHOULD use the same message-id in the article when resending it or check whether the article was successfully posted before resending it to ensure that the resend will not result in a duplicate article. Feather Expires August 30, 2003 [Page 43] Internet-Draft Network News Transport Protocol March 2003 7.3.1.3 Examples Example of a successful posting: [C] POST [S] 340 Input article; end with . [C] From: "Demo User" [C] Newsgroups: misc.test [C] Subject: I am just a test article [C] Organization: An Example Net [C] [C] This is just a test article. [C] . [S] 240 Article received OK Example of an unsuccessful posting: [C] POST [S] 340 Input article; end with . [C] From: "Demo User" [C] Newsgroups: misc.test [C] Subject: I am just a test article [C] Organization: An Example Net [C] [C] This is just a test article. [C] . [S] 441 Posting failed Example of an attempt to post when posting is not allowed: [C] MODE READER [S] 201 NNTP Service Ready, posting prohibited [C] POST [S] 440 Posting not permitted 7.3.2 IHAVE 7.3.2.1 Usage This command MUST NOT be pipelined. Syntax IHAVE message-id Responses Feather Expires August 30, 2003 [Page 44] Internet-Draft Network News Transport Protocol March 2003 Initial responses 335 Send article to be transferred 435 Article not wanted 436 Transfer not possible; try again later Subsequent responses 235 Article transferred OK 436 Transfer failed; try again later 437 Transfer rejected; do not retry Parameters message-id = Article message-id 7.3.2.2 Description The IHAVE command informs the server that the client has an article with the specified message-id. If the server desires a copy of that article a 335 response MUST be returned, instructing the client to send the entire article. If the server does not want the article (if, for example, the server already has a copy of it), a 435 response MUST be returned, indicating that the article is not wanted. Finally, if the article isn't wanted immediately but the client should retry later if possible (if, for example, another client is in the process of sending the same article to the server), a 436 response MUST be returned. If transmission of the article is requested, the client MUST send the entire article, including header and body, in the format defined above (Section 3) for multi-line responses (except that there is no initial line containing a response code). Thus a single dot (".") on a line indicates the end of the text, and lines starting with a dot in the original text have that dot doubled during transmission. The server MUST return either a 235 response, indicating that the article was successfully transferred, a 436 response, indicating that the transfer failed but should be tried again later, or a 437 response, indicating that the article was rejected. This function differs from the POST command in that it is intended for use in transferring already-posted articles between hosts. It SHOULD NOT be used when the client is a personal news reading program, since this command indicates that the forthcoming article has already been posted at another site and is being forwarded from another host. However, the server MAY elect not to post or forward the article if after further examination of the article it deems it inappropriate to do so. Reasons for such subsequent rejection of an article may include such problems as inappropriate newsgroups or distributions, disc space limitations, article lengths, garbled Feather Expires August 30, 2003 [Page 45] Internet-Draft Network News Transport Protocol March 2003 headers, and the like. These are typically restrictions enforced by the server host's news software and not necessarily the NNTP server itself. The client SHOULD NOT assume that the article has been successfully transferred unless it receives an affirmative response from the server. A lack of response (such as a dropped network connection or a network timeout) SHOULD be treated the same as a 436 response. Because some news server software may not be able immediately to determine whether or not an article is suitable for posting or forwarding, an NNTP server MAY acknowledge the successful transfer of the article (with a 235 response) but later silently discard it. 7.3.2.3 Examples Example of successfully sending an article to another site: [C] IHAVE [S] 335 Send it; end with . [C] Path: pathost!demo!somewhere!not-for-mail [C] From: "Demo User" [C] Newsgroups: misc.test [C] Subject: I am just a test article [C] Date: 6 Oct 1998 04:38:40 -0500 [C] Organization: An Example Com, San Jose, CA [C] Message-ID: [C] [C] This is just a test article. [C] . [S] 235 Article transferred OK Example of sending an article to another site that rejects it: [C] IHAVE [S] 335 Send it; end with . [C] Path: pathost!demo!somewhere!not-for-mail [C] From: "Demo User" [C] Newsgroups: misc.test [C] Subject: I am just a test article [C] Date: 6 Oct 1998 04:38:40 -0500 [C] Organization: An Example Com, San Jose, CA [C] Message-ID: [C] [C] This is just a test article. [C] . [S] 437 Article rejected; don't send again Feather Expires August 30, 2003 [Page 46] Internet-Draft Network News Transport Protocol March 2003 Example of sending an article to another site where the transfer fails: [C] IHAVE [S] 335 Send it; end with . [C] Path: pathost!demo!somewhere!not-for-mail [C] From: "Demo User" [C] Newsgroups: misc.test [C] Subject: I am just a test article [C] Date: 6 Oct 1998 04:38:40 -0500 [C] Organization: An Example Com, San Jose, CA [C] Message-ID: [C] [C] This is just a test article. [C] . [S] 436 Transfer failed Example of sending an article to a site that already has it: [C] IHAVE [S] 435 Duplicate Example of sending an article to a site that requests the article be tried again later: [C] IHAVE [S] 436 Retry later Feather Expires August 30, 2003 [Page 47] Internet-Draft Network News Transport Protocol March 2003 8. Information commands This section lists other commands that may be used at any time between the beginning of a session and its termination. Using these commands does not alter any state information, but the response generated from their use may provide useful information to clients. All servers MUST implement these commands. 8.1 DATE 8.1.1 Usage Syntax DATE Responses 111 yyyymmddhhmmss server date and time Parameters yyyymmddHHmmss = Current UTC date and time on server 8.1.2 Description This command exists to help clients find out the current Coordinated Universal Time [9] from the server's perspective. This command MUST NOT be used as a substitute for NTP [10], but to provide information that might be useful when using the NEWNEWS command (see Section 8.4). A system providing NNTP service SHOULD implement NTP for the purposes of keeping the system clock as accurate as possible. The server MUST return a 111 response specifying the date and time on the server in the form yyyymmddhhmmss. This date and time is in Coordinated Universal Time. 8.1.3 Examples [C] DATE [S] 111 19990623135624 8.2 HELP 8.2.1 Usage Feather Expires August 30, 2003 [Page 48] Internet-Draft Network News Transport Protocol March 2003 Syntax HELP Responses 100 Help text follows (multiline) 8.2.2 Description This command provides a short summary of commands that are understood by this implementation of the server. The help text will be presented as a multiline response following the 100 response code. This text is not guaranteed to be in any particular format and MUST NOT be used by clients as a replacement for the LIST EXTENSIONS command described in Section 6.1 8.2.3 Examples [C] HELP [S] 100 Help text follows [S] This is some help text. There is no specific [S] formatting requirement for this test, though [S] it is customary for it to list the valid commands [S] and give a brief definition of what they do [S] . 8.3 NEWGROUPS 8.3.1 Usage Syntax NEWGROUPS date time [GMT] Responses 231 List of new newsgroups follows (multiline) Parameters date = Date in yymmdd or yyyymmdd format time = Time in hhmmss format 8.3.2 Description This command returns a list of newsgroups created on the server since the specified date and time. The results are in the same format as the LIST ACTIVE command (see Section 8.6.1). Feather Expires August 30, 2003 [Page 49] Internet-Draft Network News Transport Protocol March 2003 OUTSTANDING ISSUE Does the output include high/low/status or not? If so, the examples are wrong. If not, the above text is wrong. The date is specified as 6 or 8 digits in the format [xx]yymmdd, where xx is the first two digits of the year (19-99), yy is the last two digits of the year (00-99), mm is the month (01-12), and dd is the day of the month (01-31). If the first two digits of the year are not specified, the year is to be taken from the current century if yy is smaller than or equal to the current year, otherwise the year is from the previous century. The time is specified as 6 digits in the format hhmmss, where hh is the hours in the 24-hour clock (00-23), mm is the minutes (00-59), and ss is the seconds (00-60, to allow for leap seconds). The token "GMT" specifies that the date and time are given in Coordinated Universal Time; if it is omitted then the date and time are specified in the server's local timezone. Note that there is no way using the protocol specified in this document to establish the server's local timezone. Note that an empty list is a possible valid response and indicates that there are no new newsgroups since that date-time. Clients SHOULD make all queries using Coordinated Universal Time (i.e. by including the "GMT" parameter) when possible. 8.3.3 Examples Example where there are new groups: [C] NEWGROUPS 19990624 000000 GMT [S] 231 list of new newsgroups follows [S] alt.rfc-writers.recovery [S] tx.natives.recovery [S] . Example where there are no new groups: [C] NEWGROUPS 19990624 000000 GMT [S] 231 list of new newsgroups follows [S] . 8.4 NEWNEWS Feather Expires August 30, 2003 [Page 50] Internet-Draft Network News Transport Protocol March 2003 8.4.1 Usage Syntax NEWNEWS wildmat date time [GMT] Responses 230 List of new articles follows (multiline) Parameters wildmat = Newsgroups of interest date = Date in yymmdd or yyyymmdd format time = Time in hhmmss format 8.4.2 Description This command returns a list of message-ids of articles posted or received on the server, in the newsgroups whose names match the wildmat, since the specified date and time. One message-id is sent on each line; the order of the response has no specific significance and may vary from response to response in the same session. A message-id MAY appear more than once; if it does so, it has the same meaning as if it appeared only once. Date and time are in the same format as the NEWGROUPS command (see Section 8.3). Note that an empty list is a possible valid response and indicates that there is currently no new news in the relevant groups. Clients SHOULD make all queries in Coordinated Universal Time (i.e. by using the "GMT" parameter) when possible. 8.4.3 Examples Example where there are new articles: [C] NEWNEWS news.*,sci.* 19990624 000000 GMT [S] 230 list of new articles by message-id follows [S] [S] [S] . Example where there are no new articles: [C] NEWNEWS alt.* 19990624 000000 GMT [S] 230 list of new articles by message-id follows [S] . Feather Expires August 30, 2003 [Page 51] Internet-Draft Network News Transport Protocol March 2003 8.5 Time As described in Section 7, each article has an arrival timestamp. Each newsgroup also has a creation timestamp. These timestamps are used by the NEWNEWS and NEWGROUP commands to construct their reponses. The DATE command MUST return a timestamp from the same clock as is used for determining article arrival and group creation times. This clock SHOULD be monotonic, and adjustments SHOULD be made by running it fast or slow compared to "real" time rather than by making sudden jumps. Clients can ensure that they do not have gaps in lists of articles or groups by using the DATE command in the following manner: First session: Issue DATE command and record result Issue NEWNEWS command using a previously chosen timestamp Subsequent sessions: Issue DATE command and hold result in temporary storage Issue NEWNEWS command using timestamp saved from previous session Overwrite saved timestamp with that currently in temporary storage In order to allow for minor errors, clients MAY want to adjust the timestamp back by two or three minutes before using it in NEWNEWS. 8.5.1 Examples First session: [C] DATE [S] 111 20010203112233 [C] NEWNEWS local.chat 20001231 235959 GMT [S] 230 list follows [S] [S] [S] [S] . Second session (the client has subtracted 3 minutes from the timestamp returned previously): [C] DATE [S] 111 20010204003344 [C] NEWNEWS local.chat 20010203 111933 GMT [S] 230 list follows Feather Expires August 30, 2003 [Page 52] Internet-Draft Network News Transport Protocol March 2003 [S] [S] [S] [S] . Note how arrived in the 3 minute gap and so is listed in both responses. 8.6 The LIST commands 8.6.1 LIST ACTIVE 8.6.1.1 Usage Syntax LIST ACTIVE [wildmat] Responses 215 Information follows (multiline) Parameters wildmat = groups of interest 8.6.1.2 Description The LIST ACTIVE command with no parameters returns a list of valid newsgroups and associated information. Each newsgroup is sent as a line of text in the following format: group first last status where: "group" is the name of the newsgroup; "first" is the current low water mark for the group; "last" is the current high water mark for the group; "status" is the current status of the group on this server; typically this is one of: "y" posting is permitted "n" posting is not permitted Feather Expires August 30, 2003 [Page 53] Internet-Draft Network News Transport Protocol March 2003 "m" postings will be forwarded to the newsgroup moderator Other status strings may exist. The definition of these other values and the circumstances under which they are returned is covered in other specifications. OUTSTANDING ISSUE Is the order "group first last status" or "group last first status"? The examples match the description above, but they don't match the news server I have tested. Each field in the line is separated from its neighboring fields by one or more US-ASCII spaces. The "first" and "last" fields correspond to the high and low water marks described in the GROUP command (see Section 7.1.1). The status of a newsgroup only indicates how posts to that newsgroup are processed. It does not indicate if the current client is permitted to post. That is indicated by the status code returned as part of the greeting. Note that an empty list is a possible valid response, and indicates that there are currently no valid newsgroups. If the optional wildmat parameter is specified, the list is limited to only the groups whose names match the wildmat. If no wildmat is specified, the keyword ACTIVE MAY be omitted without altering the effect of the command. 8.6.1.3 Examples Example of LIST ACTIVE returning a list of newsgroups: [C] LIST ACTIVE [S] 215 list of newsgroups follows [S] misc.test 3000234 3002322 y [S] alt.fc-writers.recovery 1 4 y [S] tx.natives.recovery 56 89 y [S] . Example of LIST ACTIVE omitting the second keyword and returning no newsgroups: [C] LIST [S] 215 list of newsgroups follows [S] . Example of LIST ACTIVE with a wildmat: Feather Expires August 30, 2003 [Page 54] Internet-Draft Network News Transport Protocol March 2003 [C] LIST ACTIVE *.recovery [S] 215 list of newsgroups follows [S] alt.fc-writers.recovery 1 4 y [S] tx.natives.recovery 56 89 y [S] . 8.6.2 LIST ACTIVE.TIMES 8.6.2.1 Usage This command is optional. Syntax LIST ACTIVE.TIMES [wildmat] Responses 215 Information follows (multiline) 503 Facility not available Parameters wildmat = groups of interest 8.6.2.2 Description The active.times file is maintained by some news transport systems to contain information about who created a particular newsgroup and when. Each line of this file consists of three fields separated from each other by one or more US-ASCII space characters. The first field is the name of the newsgroup. The second is the time when this group was created on this news server, measured in seconds since the start of January 1, 1970. The third is the email address of the entity that created the newsgroup, and must be a mailbox as defined in RFC 2822 [7]. If the information is available, it is returned as a multi-line response following the 215 response code. If the information is not available, a 503 response MUST be returned. If the server does not recognize the command, a 501 response MUST be returned. If the optional wildmat parameter is specified, the list is limited to only the groups whose names match the wildmat (and therefore may be empty). 8.6.2.3 Examples Example of LIST ACTIVE.TIMES returning a list of newsgroups: Feather Expires August 30, 2003 [Page 55] Internet-Draft Network News Transport Protocol March 2003 [C] LIST ACTIVE.TIMES [S] 215 information follows [S] misc.test 930445408 [S] alt.rfc-writers.recovery 930562309 [S] tx.natives.recovery 930678923 [S] . Example of LIST ACTIVE.TIMES returning an error where the command is recognised but the software does not maintain this information: [C] LIST ACTIVE.TIMES [S] 503 program error, function not performed Example of LIST ACTIVE.TIMES sent to a server that does not recognize this command: [C] LIST ACTIVE.TIMES [S] 501 Syntax Error 8.6.3 LIST DISTRIBUTIONS 8.6.3.1 Usage This command is optional. Syntax LIST DISTRIBUTIONS Responses 215 Information follows (multiline) 503 Facility not available 8.6.3.2 Description The distributions file is maintained by some news transport systems to contain information about valid values for the Distribution: line in a news article header and about what the values mean. Each line of this file consists of two fields separated from each other by one or more US-ASCII space characters. The first field is a value and the second is a short explanation of the meaning of that value. If the information is available, it is returned as a multi-line response following the 215 response code. If the information is not available, a 503 response MUST be returned. If the server does not recognize the command, a 501 response MUST be returned. Feather Expires August 30, 2003 [Page 56] Internet-Draft Network News Transport Protocol March 2003 8.6.3.3 Examples Example of LIST DISTRIBUTIONS returning a list of distributions: [C] LIST DISTRIBUTIONS [S] 215 information follows [S] usa United States of America [S] na North America [S] world All over the World [S] . Example of LIST DISTRIBUTIONS returning an error where the command is recognised but the software does not maintain this information: [C] LIST DISTRIBUTIONS [S] 503 program error, function not performed Example of LIST DISTRIBUTIONS sent to a server that does not recognize this command: [C] LIST DISTRIBUTIONS [S] 501 Syntax Error 8.6.4 LIST DISTRIB.PATS 8.6.4.1 Usage This command is optional. Syntax LIST DISTRIB.PATS Responses 215 Information follows (multiline) 503 Facility not available 8.6.4.2 Description The distrib.pats file is maintained by some news transport systems to choose a value for the Distribution: line in the header of a news article being posted. Each line of this file consists of three fields separated from each other by a US-ASCII colon. The first field is a weight, the second field is a wildmat (which may be a simple group name), and the third field is a value for the Distribution: header. Feather Expires August 30, 2003 [Page 57] Internet-Draft Network News Transport Protocol March 2003 The client MAY use this information to select a Distribution: value based on the name of a newsgroup. To do so, it should determine the lines whose second field matches the newsgroup name, select from among them the line with the highest weight (with 0 being the lowest), and use the value of the third field to construct the Distribution: header. If the information is available, it is returned as a multi-line response following the 215 response code. If the information is not available, a 503 response MUST be returned. If the server does not recognize the command, a 501 response MUST be returned. 8.6.4.3 Examples Example of LIST DISTRIB.PATS returning a list of newsgroups: [C] LIST DISTRIB.PATS [S] 215 information follows [S] 10:local.*:local [S] 5:*:world [S] 20:local.here.*:thissite [S] . Example of LIST DISTRIB.PATS returning an error where the command is recognised but the software does not maintain this information: [C] LIST DISTRIB.PATS [S] 503 program error, function not performed Example of LIST DISTRIB.PATS sent to a server that does not recognize this command: [C] LIST DISTRIB.PATS [S] 501 Syntax Error 8.6.5 LIST NEWSGROUPS 8.6.5.1 Usage This command is optional. Syntax LIST NEWSGROUPS [wildmat] Responses 215 Information follows (multiline) 503 Facility not available Feather Expires August 30, 2003 [Page 58] Internet-Draft Network News Transport Protocol March 2003 Parameters wildmat = groups of interest 8.6.5.2 Description The newsgroups file is maintained by some news transport systems to contain the name of each newsgroup that is available on the server and a short description about the purpose of the group. Each line of this file consists of two fields separated from each other by one or more US-ASCII space characters. The first field is the name of the newsgroup and the second is a short description of the group. Note that an empty list is a possible valid response, and indicates that there are currently no valid newsgroups. If the information is available, it is returned as a multi-line response following the 215 response code. If the information is not available, a 503 response MUST be returned. If the server does not recognize the command, a 501 response MUST be returned. If the optional wildmat parameter is specified, the list is limited to only the groups whose names match the wildmat. 8.6.5.3 Examples Example of LIST NEWSGROUPS returning a list of newsgroups: [C] LIST NEWSGROUPS [S] 215 information follows [S] misc.test General Usenet testing [S] alt.rfc-writers.recovery RFC Writers Recovery [S] tx.natives.recovery Texas Natives Recovery [S] . Example of LIST NEWSGROUPS returning an error where the command is recognised but the software does not maintain this information: [C] LIST NEWSGROUPS [S] 503 program error, function not performed Example of LIST NEWSGROUPS sent to a server that does not recognize this command: [C] LIST NEWSGROUPS [S] 501 Syntax error Feather Expires August 30, 2003 [Page 59] Internet-Draft Network News Transport Protocol March 2003 9. The CONCLUSION step 9.1 QUIT 9.1.1 Usage Syntax QUIT Responses 205 Connection closing 9.1.2 Description The server process MUST acknowledge the QUIT command and then close the connection to the client. This is the preferred method for a client to indicate that it has finished all its transactions with the NNTP server. If a client simply disconnects (or the connection times out or some other fault occurs), the server MUST gracefully cease its attempts to service the client, disconnecting from its end if necessary. 9.1.3 Examples [C] QUIT [S] 205 closing connection [Server closes connection.] Feather Expires August 30, 2003 [Page 60] Internet-Draft Network News Transport Protocol March 2003 10. Framework for NNTP extensions Although NNTP is widely and robustly deployed, some parts of the Internet community might wish to extend the NNTP service. This document defines a means whereby an extended NNTP client can query the server to determine the service extensions that it supports. It must be emphasized that any extension to the NNTP service should not be considered lightly. NNTP's strength comes primarily from its simplicity. Experience with many protocols has shown that: Protocols with few options tend towards ubiquity, whilst protocols with many options tend towards obscurity. This means that each and every extension, regardless of its benefits, must be carefully scrutinized with respect to its implementation, deployment, and interoperability costs. In many cases, the cost of extending the NNTP service will likely outweigh the benefit. Given this environment, the framework for extensions described in this document consists of: o a mechanism for clients to determine a server's available extensions o a registry of NNTP service extensions The LIST EXTENSIONS command is described in this document (see Section 6.1) and is the mechanism for clients to use to determine what extensions are available. The IANA shall maintain a registry of NNTP service extensions. An extension is identified by a unique extension-label, which is a string of 1 to 12 uppercase letters. The extension-label will often be the name of a new command that the extension adds. However this is not a requirement: an extension might not add any new commands or keywords. An extension is either a private extension or else it is included in the IANA registry and is defined in an RFC. Such RFCs either must be on the standards-track or must define an IESG-approved experimental protocol. The definition of an extension must include: o a descriptive name for the extension Feather Expires August 30, 2003 [Page 61] Internet-Draft Network News Transport Protocol March 2003 o the extension-label (which is returned by LIST EXTENSIONS to indicate to the client that the server supports this particular extension) o the syntax, values, and meanings of any parameters following the extension-label in the output of LIST EXTENSIONS o any new NNTP commands associated with the extension o the syntax and possible values of parameters associated with the new NNTP commands o any new parameters the extension associates with any other pre-existing NNTP commands o how support for the extension affects the behavior of a server and NNTP client o any increase in the maximum length of commands over the value specified in this document o a specific statement about the effect on pipelining this extension may have (if any) The extension-label of private extensions MUST begin with "X". The extension-label of registered extensions MUST NOT begin with "X". A server MUST NOT provide any extension, whether or not listed in the output from LIST EXTENSIONS, unless it is either a registered extension or a private extension. Except where stated otherwise, the commands in this document are understood (even if not supported) by all servers and are not described in the list of features returned by the LIST EXTENSIONS command. A server MAY provide additional keywords - either for new commands or new variants of existing commands - as part of a private extension. These new keywords MUST begin with "X". A server MUST NOT send different response codes to basic NNTP commands documented here or commands documented in registered extensions in response to the availability or use of a private extension. Feather Expires August 30, 2003 [Page 62] Internet-Draft Network News Transport Protocol March 2003 10.1 Initial IANA registry The IANA's initial registry of NNTP service extensions consists of these entries: Extension Label Added behavior Specific article numbers LISTGROUP Defined in this document Overview support OVER Defined in this document Header pattern matching HDR Defined in this document 10.2 Standard extensions Each of the following sections describes an extension that a server MAY provide. If the server provides the extension, it MUST include the appropriate extension label in the response to LIST EXTENSIONS. If it does not provide it, it MUST NOT include the appropriate extension label. The descriptions of facilities in each section are written as if the extension is provided. If it is not provided, the entire section should be ignored. If the server provides an extension, it MUST implement all of the commands in the specification of the extension except for those marked as optional. If it does not provide an extension, it MUST NOT implement any of the commands in the specification of that extension. 10.3 The LISTGROUP extension This extension provides one command and has the extension label LISTGROUP. 10.3.1 LISTGROUP 10.3.1.1 Usage Syntax LISTGROUP [ggg] Responses 211 Article numbers follow (multiline) 411 No such newsgroup 412 No newsgroup selected [1] Parameters ggg = name of newsgroup Feather Expires August 30, 2003 [Page 63] Internet-Draft Network News Transport Protocol March 2003 [1] The 412 response can only occur if no group has been specified. 10.3.1.2 Description The LISTGROUP command is used to get a listing of all the article numbers in a particular newsgroup. The optional parameter ggg is the name of the newsgroup to be selected (e.g. "news.software.misc"). A list of valid newsgroups may be obtained from the LIST ACTIVE command. If no group is specified, the current selected newsgroup is used. OUTSTANDING ISSUE On at least some servers the 211 response line is the same as with GROUP. Should this be a requirement? The list of article numbers is returned as a multi-line response following the 211 response code. It contains one number per line, is in numerical order, and lists precisely those articles that exist in the group. When a valid group is selected by means of this command, the current selected newsgroup MUST be set to that group and the current article number MUST be set to the first article in the group. If an empty newsgroup is selected, the current article pointer is made invalid. If an invalid group is specified, the current selected newsgroup and current article number MUST NOT be changed. The LISTGROUP command MAY be used by a client as a replacement for the GROUP command in establishing a valid current selected newsgroup and current article number. If the group specified is not available on the server, a 411 response MUST be returned. If no group is specified and the current selected newsgroup is invalid, a 412 response MUST be returned. 10.3.1.3 Examples Example of LISTGROUP on an empty group: [C] LISTGROUP example.empty.newsgroup [S] 211 list of article numbers follows [S] . Example of LISTGROUP on a valid current selected newsgroup: Feather Expires August 30, 2003 [Page 64] Internet-Draft Network News Transport Protocol March 2003 [C] GROUP misc.test [S] 211 2000 3000234 3002322 misc.test selected [C] LISTGROUP [S] 211 list follows [S] 3000234 [S] 3000237 [S] 3000238 [S] 3000239 [S] 3002322 [S] . Example of LISTGROUP failing because no group has been selected: [Assumes current selected newsgroup is invalid.] [C] LISTGROUP [S] 412 no current group [C] GROUP example.is.sob.bradner.or.barber [S] 411 no such group [C] LISTGROUP [S] 412 no current group 10.4 Article metadata The OVER and HDR extensions refer to the concept of "article metadata". This is data about articles that does not occur within the article itself. Each metadata item has a name which MUST begin with a colon. Note that a historical feature of the LIST OVERVIEW.FMT command means that metadata names SHOULD NOT end with ":full". When generating a metadata item, the server MUST compute it for itself and MUST NOT trust any related value provided in the article. (In particular, a Lines: or Bytes: header in the article MUST NOT be assumed to specify the correct number of lines or bytes in the article.) This specification defines two metadata items: ":bytes" and ":lines". Implementations and other extensions may define other metadata items. 10.4.1 The :bytes metadata item The :bytes metadata item for an article is a decimal integer. It MUST equal the number of octets in the entire article - headers, body, and separating blank line - except that the US-ASCII CRLF at the end of each line MAY (but SHOULD NOT) be counted as a single octet. Feather Expires August 30, 2003 [Page 65] Internet-Draft Network News Transport Protocol March 2003 OUTSTANDING ISSUE Should this be called ":octets" instead? Or should it be a count of UTF characters rather than octets? 10.4.2 The :lines metadata item The :lines metadata item for an article is a decimal integer. It MUST equal the number of lines in the article body (excluding the blank line separating headers and body); equivalently, it is two less than the number of US-ASCII CRLF pairs that the BODY command would return for that article (the extra two are those following the response code and the termination octet). 10.5 The OVER extension This extension provides two commands, OVER and LIST OVERVIEW.FMT. The label for this extension is OVER. The OVER extension provides access to the overview database [8], which is a database of header lines extracted from incoming articles. Only certain headers are included in the database. The database also includes some article metadata. The information stored in the database may change over time. The LIST OVERVIEW.FMT command describes the information that would be stored for an article arriving at the same time as the command was executed. 10.5.1 OVER 10.5.1.1 Usage Syntax OVER [range] Responses 224 Overview information follows (multiline) 412 No newsgroup selected 420 Current article number is invalid 423 No articles in that range Parameters range = Article(s) to return information for Feather Expires August 30, 2003 [Page 66] Internet-Draft Network News Transport Protocol March 2003 10.5.1.2 Description The OVER command returns the contents of the headers and metadata in the database for the article(s) specified from the current selected newsgroup. The optional range argument may be any of the following: o an article number o an article number followed by a dash to indicate all following o an article number followed by a dash followed by another article number If no argument is specified, then the current article number is used. If the information is available, it is returned as a multi-line response following the 224 response code. If the current selected newsgroup is invalid, a 412 response MUST be returned. If there are no articles in the range specified, a 423 response MUST be returned. If OVER is sent without any arguments and the current article number is invalid, a 420 response MUST be returned. If the client does not have permission to access the overview database, a 502 response MUST be returned. OUTSTANDING ISSUE Should this be 502 ("not permitted") or 503 ("there is no overview database")? In which case, why provide the command? For a successful response, the output consists of one line per article, sorted in numerical order of article number. Each line consists of a number of fields separated by an US-ASCII TAB character. A field may be empty (in which case there will be two adjacent US-ASCII TABs), and a sequence of trailing US-ASCII TABs may be omitted. The first 8 fields MUST be the following, in order: article number "Subject" header "From" header "Date" header "Message-ID" header "References" header :bytes metadata item :lines metadata item Feather Expires August 30, 2003 [Page 67] Internet-Draft Network News Transport Protocol March 2003 Any subsequent fields are the contents of the other headers and metadata held in the database. For the five mandatory headers, the content of each field MUST be based on the original header with the header name and following colon and space removed. If the article does not contain that header, or if there is nothing following the colon and space, the field MUST be empty. For the two mandatory metadata items, the content of the field MUST be just the value, with no other text. For all subsequent fields that contain headers, the content MUST be based on the entire header including the name. For all subsequent fields that contain metadata, the field consists of the metadata name, a single US-ASCII space, and then the value. For all fields, the value is processed by first removing all US-ASCII CRLF pairs and then replacing each remaining US-ASCII NUL, TAB, CR, or LF character with a single US-ASCII space (for example, CR LF LF TAB will become two spaces). If there is no such header in the article, or no such metadata item, or no header or item stored in the database for that article, the corresponding field MUST be empty. The server SHOULD NOT produce output for articles that no longer exist. 10.5.1.3 Examples In the first two examples, US-ASCII tab has been replaced by vertical bar and some lines have been folded for readability. Example of a successful retrieval of overview information for an article (using no article number): [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] OVER [S] 224 Overview information follows [S] 300234|I am just a test article|"Demo User" |6 Oct 1998 04:38:40 -0500| <45223423@example.com>|<45454@example.net>|1234| 17|Xref: news.example.com misc.test:3000363 [S] . Example of a successful retrieval of overview information for a range of articles: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test Feather Expires August 30, 2003 [Page 68] Internet-Draft Network News Transport Protocol March 2003 [C] OVER 3000234-3000240 [S] 224 Overview information follows [S] 300234|I am just a test article|"Demo User" |6 Oct 1998 04:38:40 -0500| <45223423@example.com>|<45454@example.net>|1234| 17|Xref: news.example.com misc.test:3000363 [S] 3000235|Another test article|nobody@nowhere.to (Demo User)|6 Oct 1998 04:38:45 -0500|<45223425@to.to>|| 4818|37||Distribution: fi [S] 3000238|Re: I am just a test article|somebody@elsewhere.to| 7 Oct 1998 11:38:40 +1200|| <45223423@to.to>|9234|51 [S] . Note the missing "References" and Xref headers in the second line, the missing trailing field(s) in the first and last lines, and that there are only results for those articles that still exist. Example of an unsuccessful retrieval of overview information on an article by number: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] OVER 300256 [S] 420 No such article in this group Example of an unsuccessful retrieval of overview information by number because no newsgroup was selected first: [Assumes current selected newsgroup is invalid.] [C] OVER [S] 412 No newsgroup selected Example of an attempt to retrieve information when the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] OVER [S] 420 No current article selected 10.5.2 LIST OVERVIEW.FMT 10.5.2.1 Usage Feather Expires August 30, 2003 [Page 69] Internet-Draft Network News Transport Protocol March 2003 Syntax LIST OVERVIEW.FMT Responses 215 Information follows (multiline) 503 Facility not available 10.5.2.2 Description OUTSTANDING ISSUE Should this be optional even when the OVER extension is provided? If so, is there a point in the 503 response? The LIST OVERVIEW.FMT command returns a description of the fields in the database. The fields MUST be listed in the order that they will be returned by the OVER command for a newly-received article (the information stored for articles may change over time). If the information is available, it is returned as a multi-line response following the 215 response code. If the information is not available, a 503 response MUST be returned. The information contains one line per field in the order they are returned by the OVER command; he first 7 lines MUST be exactly: Subject: From: Date: Message-ID: References: :bytes :lines except that, for compatibility with existing implementations, the last two lines MAY instead be: Bytes: Lines: even though they refer to metadata, not headers. All subsequent lines MUST consist of either a header name followed by ":full", or the name of a piece of metadata. There are no leading or trailing spaces in the output. Note that the 7 fixed lines describe the 2nd to 8th fields of the Feather Expires August 30, 2003 [Page 70] Internet-Draft Network News Transport Protocol March 2003 OVER output. The "full" suffix is a reminder that the corresponding fields include the header name. This command MAY generate different results if used more than once in a session. 10.5.2.3 Examples Example of LIST OVERVIEW.FMT output corresponding to the example OVER output above, using the preferred format: [C] LIST OVERVIEW.FMT [S] 215 Order of fields in overview database. [S] Subject: [S] From: [S] Date: [S] Message-ID: [S] References: [S] :bytes [S] :lines [S] Xref:full [S] Distribution:full [S] . Example of LIST OVERVIEW.FMT output corresponding to the example OVER output above, using the alternative format: [C] LIST OVERVIEW.FMT [S] 215 Order of fields in overview database. [S] Subject: [S] From: [S] Date: [S] Message-ID: [S] References: [S] Bytes: [S] Lines: [S] Xref:full [S] Distribution:full [S] . Example of LIST OVERVIEW.FMT returning an error: [C] LIST OVERVIEW.FMT [S] 503 overview.fmt not available Feather Expires August 30, 2003 [Page 71] Internet-Draft Network News Transport Protocol March 2003 10.6 The HDR extension This extension provides one new command: HDR. The label for this extension is HDR. 10.6.1 HDR 10.6.1.1 Usage Syntax HDR header range HDR header message-id HDR header Responses First form (range specified) 225 Headers follow (multiline) 412 No newsgroup selected 423 No articles in that range Second form (message-id specified) 225 Headers follow (multiline) 430 No article with that message-id Third form (current article number used) 225 Headers follow (multiline) 412 No newsgroup selected 420 Current article number is invalid Parameters header = name of header, without the colon range = number(s) of articles message-id = message-id of article 10.6.1.2 Description The HDR command retrieves specific headers from an article or specified range of articles in the current selected newsgroup, or from an article specified by message-id. It can also return certain metadata about the article or articles. The required header parameter is the name of a header (e.g. "subject") in an article, or the name of a metadata item, and is case-insensitive. See RFC 1036 [6] for a list of valid header lines. Names of metadata items always include a colon. Except where stated otherwise, metadata items are treated as if they were header values, Feather Expires August 30, 2003 [Page 72] Internet-Draft Network News Transport Protocol March 2003 and references to headers in this description apply equally to metadata items. OUTSTANDING ISSUE Should this be changed to require the name to *begin* with a colon? The range parameter may be any of the following: o an article number o an article number followed by a dash to indicate all following o an article number followed by a dash followed by another article number The message-id argument indicates a specific article. As shown by the syntax, the range and message-id arguments are mutually exclusive; if neither are specified, the current article number is used. If the information is available, it is returned as a multi-line response following the 225 response code and contains one line for each article where the relevant header line exists. The line consists of the article number, a US-ASCII space, and then the contents of the header (without the header name or the colon and space that follow it) or metadata item. If the article is specified by message-id rather than by article range, the article number is given as "0". Header contents are modified as follows: all US-ASCII CRLF pairs are removed, and then each remaining US-ASCII NUL, TAB, CR, or LF character is replaced with a single US-ASCII space. (Note that this is the same transformation as is performed by the OVER extension.) The header content is in all cases taken from the article. This means that, for example, a request for the header "Lines" returns the contents of the "Lines" header of the specified articles, if any, not the line count metadata or any other server-generated value. If the header occurs in a given article multiple times, only the value of the first occurrence is returned by HDR. If the requested header is not present in the article or if it is present but empty, a line for that article is included in the output but the header content portion of the line is empty (the space after the article number MAY be retained or omitted). If any article number in the provided range does not exist in the group, no line for Feather Expires August 30, 2003 [Page 73] Internet-Draft Network News Transport Protocol March 2003 that article number is included in the output. If the optional argument is a message-id and no such article exists, a 430 response MUST be returned. If the optional argument is not a message-id and the current selected newsgroup is invalid, a 412 response MUST be returned. If the optional argument is an article number or number range and no article with that number or in that number range exists in the current selected newsgroup, a 423 response MUST be returned. If HDR is sent without any arguments and the current article number is invalid, a 420 response MUST be returned. A server MAY only allow HDR commands for a limited set of headers and metadata items (such as those present in the overview database). If so, it MUST respond with a 503 response to attempts to request other headers, rather than returning erroneous results such as a successful empty response. 10.6.1.3 Examples Example of a successful retrieval of subject lines from a range of articles (3000235 has no Subject header, and 3000236 is missing): [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] HDR Subject 3000234-300238 [S] 225 Headers follow [S] 3000234 I am just a test article [S] 3000235 [S] 3000237 Re: I am just a test article [S] 3000238 Ditto [S] . Example of a successful retrieval of line counts from a range of articles: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] HDR :lines 3000234-300238 [S] 225 Headers follow [S] 3000234 42 [S] 3000235 5 [S] 3000237 11 [S] 3000238 2378 [S] . Example of a successful retrieval of the subject line from an article by message-id: [C] GROUP misc.test Feather Expires August 30, 2003 [Page 74] Internet-Draft Network News Transport Protocol March 2003 [S] 211 1234 3000234 3002322 misc.test [C] HDR subject [S] 225 Header information follows [S] 0 I am just a test article [S] . Example of a successful retrieval of the subject line from the current article: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] HDR subject [S] 225 Header information follows [S] 3000234 I am just a test article [S] . Example of an unsuccessful retrieval of a header from an article by message-id: [C] HDR subject [S] 430 No Such Article Found Example of an unsuccessful retrieval of headers from articles by number because no newsgroup was selected first: [Assumes current selected newsgroup is invalid.] [C] HDR subject 300256- [S] 412 No newsgroup selected Example of an unsuccessful retrieval of headers because the current selected newsgroup is empty: [C] GROUP example.empty.newsgroup [S] 211 0 0 0 example.empty.newsgroup [C] HDR subject 1- [S] 423 No articles in that range Example of an unsuccessful retrieval of headers because the server does not allow HDR commands for that header: [C] GROUP misc.test [S] 211 1234 3000234 3002322 misc.test [C] HDR Content-Type 3000234-300238 [S] 503 HDR not permitted on Content-Type Feather Expires August 30, 2003 [Page 75] Internet-Draft Network News Transport Protocol March 2003 11. Augmented BNF Syntax for NNTP Commands This syntax defines the non-terminal "command-line". Note that ABNF strings are case insensitive. command-line = command EOL command = article-command / body-command / date-command / group-command / hdr-command / head-command / help-command / ihave-command / last-command / list-active-command / list-active-times-command / list-distrib-pats-command / list-distributions-command / list-extensions-command / list-newsgroups-command / list-overview-fmt-command / listgroup-command / mode-reader-command / newgroups-command / newnews-command / next-command / over-command / post-command / quit-command / stat-command / x-command article-command = "ARTICLE" [article-ref] body-command = "BODY" [article-ref] date-command = "DATE" group-command = "GROUP" WS newsgroup-name hdr-command = "HDR" WS header-meta-name [range-ref] head-command = "HEAD" [article-ref] help-command = "HELP" ihave-command = "IHAVE" WS message-id last-command = "LAST" list-active-command = "LIST" [WS "ACTIVE" [WS wildmat]] list-active-times-command = "LIST" WS "ACTIVE.TIMES" [WS wildmat] list-distrib-pats-command = "LIST" WS "DISTRIB.PATS" list-distributions-command = "LIST" WS "DISTRIBUTIONS" list-extensions-command = "LIST" WS "EXTENSIONS" list-newsgroups-command = "LIST" WS "NEWSGROUPS" [WS wildmat] list-overview-fmt-command = "LIST" WS "OVERVIEW.FMT" Feather Expires August 30, 2003 [Page 76] Internet-Draft Network News Transport Protocol March 2003 listgroup-command = "LISTGROUP" [WS newsgroup-name] mode-reader-command = "MODE" WS "READER" newgroups-command = "NEWGROUPS" WS date-time newnews-command = "NEWNEWS" WS wildmat WS date-time next-command = "NEXT" over-command = "OVER" [WS range] post-command = "POST" quit-command = "QUIT" stat-command = "STAT" [article-ref] x-command = x-command-name *(WS x-argument) ; Each extension command is specified fully elsewhere article-ref = WS (article-number / message-id) article-number = 1*16DIGIT date = [2DIGIT] 6DIGIT date-time = date WS time [WS "GMT"] header-meta-name = header-name / metadata-name header-name = 1*header-name-char header-name-char = %x21-39 / %x3B-7E ; exclude SP and : message-id = "<" 1*248message-id-char ">" ; subject to requirements in Section 7 > message-id-char = %x21-3B / %x3C / %x3E-7E ; exclude SP < > metadata-name = ":" 1*header-name-char newsgroup-name = 1*wildmat-exact range = article-number ["-" [article-number]] range-ref = WS (range / message-id) time = 6DIGIT x-command-name = 3*12%x21-7E x-argument = 1*(%x21-7E / UTF-8-non-ascii) wildmat = wildmat-pattern *("," ["!"] wildmat-pattern) wildmat-pattern = 1*wildmat-item wildmat-item = wildmat-exact / wildmat-wild wildmat-exact = %x21-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E / UTF-8-non-ascii ; exclude * , ? [ \ ] wildmat-wild = "*" / "?" CR = %x0D CRLF = CR LF DIGIT = %x30-39 EOL = *(SP / HT) CRLF HT = %x09 LF = %x0A SP = %x20 UTF-8-non-ascii = UTF8-2 / UTF8-3 / UTF8-4 / UTF8-5 / UTF8-6 UTF8-1 = %x80-BF UTF8-2 = %xC2-DF UTF8-1 UTF8-3 = %xE0 %A0-BF UTF8-1 / %xE1-EC 2UTF8-1 / %xED %80-9F UTF8-1 / %xEE-EF 2UTF8-1 Feather Expires August 30, 2003 [Page 77] Internet-Draft Network News Transport Protocol March 2003 UTF8-4 = %xF0 %90-BF 2UTF8-1 / %xF1-F7 3UTF8-1 UTF8-5 = %xF8 %88-BF 3UTF8-1 / %xF9-FB 4UTF8-1 UTF8-6 = %xFC %84-BF 4UTF8-1 / %xFD 5UTF8-1 WS = 1*(SP / HT) Feather Expires August 30, 2003 [Page 78] Internet-Draft Network News Transport Protocol March 2003 12. Security Considerations This section is meant to inform application developers, information providers, and users of the security limitations in NNTP as described by this document. The discussion does not include definitive solutions to the problems revealed, though it does make some suggestions for reducing security risks. 12.1 Personal and Proprietary Information NNTP, because it was created to distribute network news articles, will forward whatever information is stored in those articles. Specification of that information is outside this scope of this document, but it is likely that some personal and/or proprietary information is available in some of those articles. It is very important that designers and implementers provide informative warnings to users so personal and/or proprietary information in material that is added automatically to articles (e.g. in headers) is not disclosed inadvertently. Additionally, effective and easily understood mechanisms to manage the distribution of news articles SHOULD be provided to NNTP Server administrators, so that they are able to report with confidence the likely spread of any particular set of news articles. 12.2 Abuse of Server Log Information A server is in the position to save session data about a user's requests that might identify their reading patterns or subjects of interest. This information is clearly confidential in nature and its handling can be constrained by law in certain countries. People using the NNTP protocol to provide data are responsible for ensuring that such material is not distributed without the permission of any individuals that are identifiable by the published results. 12.3 Weak Authentication and Access Control There is no user-based or token-based authentication in the basic NNTP specification. Access is normally controlled by server configuration files. Those files specify access by using domain names or IP addresses. However, this specification does permit the creation of extensions to the NNTP protocol itself for such purposes. While including such mechanisms is optional, doing so is strongly encouraged. Other mechanisms are also available. For example, a proxy server could be put in place that requires authentication before connecting via the proxy to the NNTP server. Feather Expires August 30, 2003 [Page 79] Internet-Draft Network News Transport Protocol March 2003 12.4 DNS Spoofing Many existing NNTP implementations authorize incoming connections by checking the IP address of that connection against the IP addresses obtained via DNS lookups of lists of domain names given in local configuration files. Servers that use this type of authentication, and clients that find a server by doing a DNS lookup of the server name, rely very heavily on the Domain Name Service, and are thus generally prone to security attacks based on the deliberate misassociation of IP addresses and DNS names. Clients and servers need to be cautious in assuming the continuing validity of an IP number/DNS name association. In particular, NNTP clients and servers SHOULD rely on their name resolver for confirmation of an IP number/DNS name association, rather than caching the result of previous host name lookups. Many platforms already can cache host name lookups locally when appropriate, and they SHOULD be configured to do so. It is proper for these lookups to be cached, however, only when the TTL (Time To Live) information reported by the name server makes it likely that the cached information will remain useful. If NNTP clients or servers cache the results of host name lookups in order to achieve a performance improvement, they MUST observe the TTL information reported by DNS. If NNTP clients or servers do not observe this rule, they could be spoofed when a previously accessed server's IP address changes. As network renumbering is expected to become increasingly common, the possibility of this form of attack will grow. Observing this requirement thus reduces this potential security vulnerability. This requirement also improves the load-balancing behavior of clients for replicated servers using the same DNS name and reduces the likelihood of a user's experiencing failure in accessing sites that use that strategy. 12.5 UTF-8 issues The UTF-8 specification [2] permits only certain sequences of octets and designates others as either malformed or "illegal". The Unicode standard identifies a number of security issues related to illegal sequences and forbids their generation by conforming implementations. Implementations of this specification MUST NOT generate malformed or illegal sequences and SHOULD detect them and take some appropriate action. This could include: o replacing such sequences by a "guessed" valid sequence (based on Feather Expires August 30, 2003 [Page 80] Internet-Draft Network News Transport Protocol March 2003 properties of the UTF-8 encoding); o replacing such sequences by the sequence %xEF.BF.BD, which encodes the "replacement character"; o closing the connection; o generating a 501 response code. Feather Expires August 30, 2003 [Page 81] Internet-Draft Network News Transport Protocol March 2003 13. Acknowledgments The author acknowledges the original authors of NNTP as documented in RFC 977: Brian Kantor and Phil Lapsey. The author gratefully acknowledges the work of the NNTP committee chaired by Eliot Lear. The organization of this document was influenced by the last available draft from this working group. A special thanks to Eliot for generously providing the original machine-readable sources for that document. The author gratefully acknowledges the work of Marshall Rose & John G. Meyers in RFC 1939 and the work of the DRUMS working group, specifically RFC 1869, which is the basis of the NNTP extensions mechanism detailed in this document. OUTSTANDING ISSUE Why RFC 1939? The author gratefully acknowledges the authors of RFC 2616 for providing specific and relevant examples of security issues that should be considered for HTTP. Since many of the same considerations exist for NNTP, those examples that are relevant have been included here with some minor rewrites. The author gratefully acknowledges the comments and additional information provided by the following individuals in preparing one or more of the progenitors of this document: Russ Allbery Wayne Davison Chris Lewis Tom Limoncelli Eric Schnoebelen Rich Salz This work was motivated by the work of various news reader authors and news server authors, which includes those listed below: Rick Adams Original author of the NNTP extensions to the RN news reader and last maintainer of Bnews Stan Barber Original author of the NNTP extensions to the news readers that are part of Bnews Feather Expires August 30, 2003 [Page 82] Internet-Draft Network News Transport Protocol March 2003 Geoff Collyer Original author of the OVERVIEW database proposal and one of the original authors of CNEWS Dan Curry Original author of the xvnews news reader Wayne Davison Author of the first threading extensions to the RN news reader (commonly called TRN) Geoff Huston Original author of ANU NEWS Phil Lapsey Original author of the UNIX reference implementation for NNTP Iain Lea Original maintainer of the TIN news reader Chris Lewis First known implementer of the AUTHINFO GENERIC extension Rich Salz Original author of INN Henry Spencer One of the original authors of CNEWS Kim Storm Original author of the NN news reader Finally, the present author gratefully acknowledges the vast amount of work put into previous drafts by the previous author: Stan Barber Feather Expires August 30, 2003 [Page 83] Internet-Draft Network News Transport Protocol March 2003 Normative References [1] Kantor, B. and P. Lapsley, "Network News Transfer Protocol", RFC 977, February 1986. [2] Yergeau, F., "UTF-8, a transformation format of ISO 10646", RFC 2279, January 1998. [3] American National Standards Institute, "Coded Character Set - 7-bit American Standard Code for Information Interchange", ANSI X3.4, 1986. [4] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [5] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997. [6] Horton, M. and R. Adams, "Standard for interchange of USENET messages", RFC 1036, December 1987. [7] Resnick, P., "Internet Message Format", RFC 2822, April 2001. [8] Robertson, R., "FAQ: Overview database / NOV General Information", January 1995. [9] International Telecommunications Union - Radio, "Glossary, ITU-R Recommendation TF.686-1", ITU-R Recommendation TF.686-1, October 1997. [10] Mills, D., "Network Time Protocol (Version 3) Specification, Implementation", RFC 1305, March 1992. Feather Expires August 30, 2003 [Page 84] Internet-Draft Network News Transport Protocol March 2003 Informative References [11] Salz, R., "Manual Page for wildmat(3) from the INN 1.4 distribution, Revision 1.10", April 1992. [12] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, June 1999. Author's Address Clive D.W. Feather Thus plc 322 Regents Park Road London N3 2QQ GB Phone: +44 20 8371 1138 Fax: +44 870 051 9937 URI: http://www.davros.org/ Feather Expires August 30, 2003 [Page 85] Internet-Draft Network News Transport Protocol March 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Feather Expires August 30, 2003 [Page 86] Internet-Draft Network News Transport Protocol March 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Feather Expires August 30, 2003 [Page 87]