INTERNET-DRAFT Charles H. Lindsey Usenet Format Working Group University of Manchester November 2006 News Article Architecture and Protocols Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. .QP Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.html. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire in May 2007. Abstract This Draft, together with its companion draft [USEFOR], are intended as standards track documents, together obsoleting RFC 1036, which itself dates from 1987. This Standard defines the architecture of Netnews systems and specifies the requirements to be met by software which originates, distributes, stores and displays Netnews articles. Backward compatibility has been a major goal of this endeavour, but where this standard and earlier documents or practices conflict, this standard should be followed. In most such cases, current practice is already compatible with these changes. A companion Best Current Practice document [USEAGE], addressing requirements which are present for Social rather than Normative reasons is in preparation. [This is the latest in the line of USEPRO drafts. However, the USEFOR Working Group is currently considering the possibility of a complete rewrite of this document.] C. H. Lindsey [Page 1] News Article Architecture and Protocols November 2006 [The use of the words "this standard" within this document when referring to itself does not imply that this draft yet has pretensions to be a standard, but rather indicates what will become the case if and when it is accepted as an RFC with the status of a proposed or draft standard.] [Remarks enclosed in square brackets and aligned with the left margin, such as this one, are not part of this draft, but are editorial notes to explain matters amongst ourselves, or to point out alternatives, or to assist the RFC Editor.] Table of Contents 1. Introduction .................................................. 4 1.1. Basic Concepts ............................................ 4 1.2. Objectives ................................................ 4 1.3. Historical Outline ........................................ 5 2. Definitions, Notations and Conventions ........................ 5 2.1. Definitions ............................................... 5 2.2. Defining the Architecture ................................. 5 2.3. Identification of news servers ............................ 6 2.4. Variant Header Fields ..................................... 8 2.5. Textual Notations ......................................... 8 3. Transport ..................................................... 9 4. Definition of new Media Types ................................. 10 4.1. Application/news-transmission ............................. 10 4.2. Message/news obsoleted .................................... 11 4.3. Application/news-groupinfo ................................ 11 4.4. Application/news-checkgroups .............................. 12 5. Control Messages .............................................. 13 5.1. Digital Signature of Header Fields ........................ 14 5.2. Group Control Messages .................................... 14 5.2.1. The 'newgroup' Control Message ........................ 15 5.2.1.1. The Body of the 'newgroup' Control Message ........ 15 5.2.1.2. Initial Articles .................................. 15 5.2.1.3. Example ........................................... 16 5.2.2. The 'rmgroup' Control Message ......................... 17 5.2.2.1. Example ........................................... 17 5.2.3. The 'mvgroup' Control Message ......................... 17 5.2.3.1. Example ........................................... 19 5.2.4. The 'checkgroups' Control Message ..................... 20 5.3. Cancel .................................................... 21 5.4. Ihave, sendme ............................................. 22 5.5. Obsolete control messages. ............................... 23 6. Duties of Various Agents ...................................... 23 6.1. General principles to be followed ......................... 24 6.2. Duties of an Injecting Agent .............................. 24 6.2.1. Proto-articles ........................................ 25 6.2.2. Procedure to be followed by Injecting Agents .......... 25 6.2.3. Procedure for Forwarding to a Moderator ............... 28 6.3. Duties of a Relaying Agent ................................ 28 6.3.1. Path Header Field Example ............................. 32 6.4. Duties of a Serving Agent ................................. 33 C. H. Lindsey [Page 2] News Article Architecture and Protocols November 2006 6.5. Duties of a Posting Agent ................................. 35 6.6. Duties of a Followup Agent ................................ 35 6.6.1. Construction of the References header field ........... 36 6.7. Duties of a Reading Agent ................................. 37 6.8. Duties of a Moderator ..................................... 37 6.9. Duties of a Gateway ....................................... 39 6.9.1. Duties of an Outgoing Gateway ......................... 40 6.9.2. Duties of an Incoming Gateway ......................... 40 6.9.3. Example ............................................... 42 7. Security and Related Considerations ........................... 43 7.1. Leakage ................................................... 43 7.2. Attacks ................................................... 44 7.2.1. Denial of Service ..................................... 44 7.2.2. Compromise of System Integrity ........................ 45 7.3. Liability ................................................. 46 8. IANA Considerations ........................................... 47 9. References .................................................... 47 9.1. Normative References ...................................... 47 9.2. Informative References .................................... 48 10. Acknowledgements ............................................. 48 11. Contact Address .............................................. 49 Appendix A - Obsolete Control Messages ............................ 49 Appendix B - Differences from the Protocols in RFC 1036 and its derivatives ....................................................... 50 Appendix C - Transitional Arrangements ............................ 50 Appendix D - Notices .............................................. 51 Appendix E - Change Log ........................................... 52 C. H. Lindsey [Page 3] News Article Architecture and Protocols November 2006 1. Introduction 1.1. Basic Concepts "Netnews" is a set of protocols for generating, storing and retrieving news "articles", as introduced in F-1.1. "Usenet" is a particular worldwide publicly accessible network based upon the Netnews protocols, with the newsgroups being organized into recognized "hierarchies". Anybody can join (it is simply necessary to negotiate an exchange of articles with one or more other participating hosts). An important characteristic of Usenet is the lack of any requirement for a central administration or for the establishment of any controlling host to manage the network. Nevertheless, administrative agencies do exists with varying degrees of authority to establish "policies" applicable to particular parts of Usenet. A "policy" is a rule intended to facilitate the smooth operation of a network by establishing parameters which restrict behaviour that, whilst technically unexceptionable, would nevertheless contravene some accepted standard of "Good Netkeeping". Since the ultimate beneficiaries of a network are its human readers, who will be less tolerant of poorly designed interfaces than mere computers, articles in breach of established policy can cause considerable annoyance to their recipients. [Could omit that last sentence, and perhaps even the whole paragraph?] 1.2. Objectives The purpose of this present standard is to define the overall architecture and the protocols to be used for Netnews in general, and for Usenet in particular, and to set standards to be followed by software that implements those protocols. The companion standard [USEFOR] sets out the canonical format of news articles exchanged between the various agents comprising that architecture. In this standard, references to individual sections in the companion [USEFOR] are prefixed with "F-". A set of hosts within a network which, by mutual arrangement, operates some variant (whether more or less restrictive) of the Netnews protocols is a "cooperating subnet". It is NOT the purpose of this standard to settle matters of policy, nor aspects of software behaviour which do not impinge upon the generation, transmission, storage and reception of articles, nor how the authority of various agencies to create such policies and to exercise control or oversight of the various parts of Usenet is established. For these purposes, a separate Best Current Practice document [USEAGE] is being provided. C. H. Lindsey [Page 4] News Article Architecture and Protocols November 2006 Nevertheless, it is assumed that such agencies with the necessary authority will exist, and tools are provided within the protocols for their use. 1.3. Historical Outline Network news originated as the medium of communication for Usenet, circa 1980. Since then, Usenet has grown explosively, and many Internet and non-Internet sites participate in it. In addition, the news technology is now in widespread use for other purposes, on the Internet and elsewhere. For an account of the earlier formats used in Netnews prior to [RFC 1036], see Henry Spencer's 1994 draft, popularly referred to as "Son of 1036" [Son-of-1036], which has recently been republished as an Informational RFC. [That is a tentative statement, which may need revision.] Although never adopted as a formal standard, [Son-of-1036] had a considerable effect on the development of Netnews and hence on these present standards, and it is hoped that we have followed its spirit and intentions. .nr H1 1 2. Definitions, Notations and Conventions 2.1. Definitions All the technical terms defined in F-1.5 are to be considered as defined also, with the same meaning, in this standard. In addition, some further terms are defined here, and in the following section. A "hierarchy" is the set of all newsgroups whose names share a first (as defined in F-3.1.4). The term "sub-hierarchy" is also used where several initial components are shared. The "semantic content" (often abbreviated to just "content" when the context is clear) of a header field is its semantic interpretation; i.e. what remains after unfolding it and removing its field name with its colon and any leading and trailing whitespace and, in the case of structured header fields only, ignoring comments and other semantically invisible items and replacing white space by a single SP. See 6.6.1 for the use of this term. 2.2. Defining the Architecture A Netnews system is a distributed database composed of agents of various types which, acting together according to the protocols defined in section 6 of this standard, causes articles to be propagated throughout the system and to be made available to its readers. The protocols ensure that all copies of a given article, wherever stored, are identical apart from those header fields defined as variant (2.4). For explaining the working of the protocols, it is convenient to define particular sub-categories of agent as follows: C. H. Lindsey [Page 5] News Article Architecture and Protocols November 2006 A "posting agent" is the software that assists posters to prepare proto-articles in compliance with [USEFOR]. The proto-article is then passed on to an "injecting agent" for final checking and injection into the news stream. If the article is not compliant, or is rejected by the injecting agent, then the posting agent informs the poster with an explanation of the error. A "reading agent" is software which presents articles to a reader. A "followup agent" is a combination of reading agent and posting agent that aids in the preparation and posting of a followup. An "injecting agent" takes the finished article from the posting agent (often via the NNTP "POST" command), performs some final checks and passes it on to a "relaying agent" for general distribution. A "relaying agent" is software which receives allegedly compliant articles from injecting agents and/or other relaying agents, and possibly passes copies on to other relaying agents and "storage agents". A "storage agent" receives an article from a relaying agent and files it in a "news database". It also provides an interface for reading agents to access the news database. A "news database" is the set of articles and related structural information stored by a storage agent and made available for access by reading agents. A "gateway" is software which receives news articles and converts them to messages of some other kind (e.g. mail to a mailing list), or vice versa; in essence it is a translating relaying agent that straddles boundaries between different methods of message exchange. The most common type of gateway connects newsgroup(s) to mailing list(s), either unidirectionally or bidirectionally, but there are also gateways between news networks using the [USEFOR] news format and those using other formats. Posting, reading and followup agents (which are usually just different services provided by the same piece of software) together comprise the "user agents" defined in F-1.5. Likewise, injecting, relaying and storage agents (which are often just different services provided by the same piece of software) together comprise the "news servers". 2.3. Identification of news servers [There are two alternative texts which have been proposed:] In order to record the passage of articles through the network, news servers need to identify themselves by means of a (F-3.1.5), which can appear in Path, Injection-Info and Xref header fields. Whatever is used in the Path header field C. H. Lindsey [Page 6] News Article Architecture and Protocols November 2006 SHOULD be used also in any Injection-Info header field (and it would be normal to use it in any Xref header field also). [Maybe that last sentence moves elsewhere.] NOTE: Such s may also be suitable for sending email to news server administrators (see [USEAGE]). [1st alternative] s can take the following forms (in decreasing order of preference): 1. A fully qualified domain name (FQDN) that SHOULD be resolvable in the DNS (whether via an A, AAAA or MX record or an equivalent CNAME), thus guaranteeing a unique identity. Ideally, it will also provide a means to contact the administrators by email (according to [RFC 2142], the forms "usenet@server" and "news@server" are common addresses for a news server administrator). 2. Some other (arbitrary) name in the form of a , and believed to be unique and registered at least with all other news servers sending articles directly to the given one. This option SHOULD NOT be used unless the earlier option is unavailable (e.g. because the server in question is not connected to the Internet), or unless it is of longstanding usage and cessation would be unduly disruptive, or unless the earlier option is provided as well. [2nd alternative] s can take the following forms (in decreasing order of preference): 1. A fully qualified domain name (FQDN) that can be resolved to an email server via an MX, A or AAAA record according to the procedures of [RFC 2821]; this guarantees that the name is unique, and makes it easy to contact the administrators if needed. 2. A fully qualified domain name (FQDN) that is guaranteed to be unique by the administrators of the domain; for instance, the uniqueness of "server.example.org" could be guaranteed by the administrator of "example.org" even if nothing is stored in the DNS for that name. 3. Some other (arbitrary) name in the form of a , and believed to be unique and registered at least with all other news-servers sending articles directly to the given one. This option SHOULD NOT be used unless the earlier options are unavailable, or unless the name is of longstanding usage and cessation would be unduly disruptive, or unless one of the earlier options is provided as well. C. H. Lindsey [Page 7] News Article Architecture and Protocols November 2006 According to [RFC 2142]], the forms "usenet@server" and "news@server" are common addresses for a news server administrator. [end of alternatives] NOTE: Although domain names are case insensitive and it is intended that s should also be so, it is customary to render them all in lowercase, since many implementations compare them case sensitively for reasons of efficiency. NOTE: A news server administrator who chooses a which turns out not to be unique (disregarding case) will have to bear the consequences. NOTE: An IP address is not permitted as a , although it may still appear in a . Since the syntax permits a colon (":" which, prior to this standard, was an alternative to the "!" delimiter) within any which takes the form of an , it would be unwise to choose as a anything composed solely from four or less hexadecimal digits. 2.4. Variant Header Fields Header fields with the variant property may differ between (or even be completely absent from) copies of the same article as stored or relayed throughout a Netnews system. The manner of the difference (or absence) MUST be as specified in this (or some future) standard. Typically, these header fields are modified as articles are propagated, or they reflect the status of the article on a particular storage agent, or cooperating group of such agents. A variant header field MAY be placed anywhere within the header fields (though placing it first is recommended). The following header fields are classified as "variant": o Path (F-3.1.5) - augmented at each relaying agent that an article passes through. o Xref (F-3.2.14) - used to keep track of the s of crossposted articles so that reading agents serviced by a particular storage agent can mark such articles as read. o Injection-Info (F-3.2.8) is also considered variant in some special situations involving reinjection (6.2 and 6.2.2). 2.5. Textual Notations This standard contains explanatory NOTEs using the following format. These may be skipped by persons interested solely in the content of the specification. The purpose of the notes is to explain why choices were made, to place them in context, or to suggest possible implementation techniques. NOTE: While such explanatory notes may seem superfluous in principle, they often help the less-than-omniscient reader grasp the purpose of the specification and the constraints involved. Given the limitations of natural language for descriptive C. H. Lindsey [Page 8] News Article Architecture and Protocols November 2006 purposes, this improves the probability that implementors and users will understand the true intent of the specification in cases where the wording is not entirely clear. "US-ASCII" is short for "the ANSI X3.4 character set" [ANSI X3.4]. US-ASCII is a 7 bit character set. Please note that this standard requires that all agents be 8 bit clean; that is, they must accept and transmit data without changing or omitting the 8th bit. Certain words, when capitalized, are used to define the significance of individual requirements. The key words "MUST", "REQUIRED", "SHOULD", "RECOMMENDED", "MAY" and "OPTIONAL", and any of those words associated with the word "NOT", are to be interpreted as described in [RFC 2119]. NOTE: A requirement imposed on a relaying or storage agent regarding some particular article should be understood as applying only if that article is actually accepted for processing (since any agent may always reject any article entirely, for reasons of site policy). Wherever the context permits, use of the masculine includes the feminine and use of the singular includes the plural, and vice versa. Throughout this standard we will give examples of various definitions, header fields and other specifications. It needs to be remembered that these samples are for the aid of the reader only and do NOT define any specification themselves. In order to prevent possible conflict with "Real World" entities and people the top level domain ".example" is used in all sample domains and addresses. The hierarchy "example.*" is also used as a sample hierarchy. Information on the ".example" top level domain is in [RFC 2606]. 3. Transport As in this standard's predecessors, the exact means used to transmit articles from one host to another is not specified. NNTP [RFC 3977] is the most common transmission method on the Internet, but much transmission takes place entirely independent of the Internet. Other methods in use include the UUCP protocol [RFC 976] extensively used in the early days of Usenet, FTP, tunneling through email using application news/transmission, downloading via satellite, tape archives, and physically delivered magnetic and optical media. Transmission paths for news articles MUST treat news articles as uninterpreted sequences of octets, excluding the values 0 (US-ASCII NUL) and 13 and 10 (US-ASCII CR and LF, which MUST ONLY appear in the combination CRLF which denotes a line separator). NOTE: this corresponds to the range of octets permitted for MIME "8bit data" [RFC 2045]. Thus raw binary data cannot be transmitted in an article body except by the use of a Content- Transfer-Encoding such as base64. C. H. Lindsey [Page 9] News Article Architecture and Protocols November 2006 In particular, transmission paths MUST convey all header fields (including body part header fields and header fields within message/rfc822 objects) intact, even if they contain octets in the range 128 to 255. Furthermore, relaying agents MUST, and other agents SHOULD, convey lines even if they exceed 998 characters in length, especially in article bodies. These requirements include the transmissiom paths between posting agents, injecting agents, relaying agents, storage agents and reading agents, but NOT the paths traversed by Netnews articles that have been gatewayed into Email (6.9.1). [At some point it will be necessary for the IMAP standards to catch up with these requirements.] 4. Definition of new Media Types This standard defines (or redefines) several new Media Types, which require to be registered with IANA as provided for in [RFC 4288]. 4.1. Application/news-transmission The Media Type "application/news-transmission" is intended for the encapsulation of complete news articles where the intention is that the recipient should then inject them into Netnews. This Application type provides one of the methods for mailing articles to moderators (see 6.2.2) and it is also the preferred method when sending to an email-to-news gateway (see 6.9.2). NOTE: The benefit of such encapsulation is that it removes possible conflict between news and email header fields and it provides a convenient way of "tunnelling" a news article through a transport medium that does not support 8bit characters. The MIME Media Type definition of "application/news-transmission" is: MIME type name: application MIME subtype name: news-transmission Required parameters: none Optional parameters: usage=moderate usage=inject usage=relay Encoding considerations: A transfer-encoding (such as Quoted- Printable or Base64) different from that of the article transmitted MAY be supplied (perhaps en route) to ensure correct transmission over some 7bit transport medium. Security considerations: A news article may be a "control message", which could have effects on the recipient host's system beyond just storage of the article. However, such control messages also occur in normal news flow, so most hosts will already be suitably defended against undesired effects. Published specification: [USEPRO] C. H. Lindsey [Page 10] News Article Architecture and Protocols November 2006 Body part: A complete article or proto-article, ready for injection into Netnews, or a batch of such articles in the batch format described in section 5.4. NOTE: It is likely that the recipient of an "application/news- transmission" will be a specialized gateway (e.g. a moderator's submission address) able to accept articles with only one of the three usage parameters "moderate", "inject" and "relay", hence the reason why they are optional, being redundant in most situations. Nevertheless, they MAY be used to signify the originator's intention with regard to the transmission, so removing any possible doubt. When the parameter "relay" is used, or implied, the body part MAY be a batch of articles to be transmitted together, in which case the batch format defined in section 5.4 MUST be used. 4.2. Message/news obsoleted The Media Type "message/news", as previously registered with IANA, is hereby declared obsolete. It was never widely implemented, and its default treatment as "application/octet-stream" by agents that did not recognize it was counter productive. The Media Type "message/rfc822" SHOULD be used in its place. 4.3. Application/news-groupinfo The "application/news-groupinfo" is used in conjunction with the "newgroup" (5.2.1) and "mvgroup" (5.2.3) control messages. The in the MUST agree with the in the "newgroup" or "mvgroup" control message. The Media Type "application/news-groupinfo" MUST NOT be used except as a part of such control messages. The "application/news-groupinfo" body part contains brief information about a newsgroup, i.e. the group's name, it's and the . NOTE: The presence of the "For your newsgroups file:" is intended to make the whole newgroup message compatible with current practice as described in [Son-of-1036]. The MIME Media Type definition of "application/news-groupinfo" is: MIME type name: application MIME subtype name: news-groupinfo Required parameters: none Disposition: by default, inline Encoding considerations: "7bit" or "8bit" is sufficient and MUST be used to maintain compatibility. Security considerations: this type MUST NOT be used except as part of a control message for the creation or modification of a Netnews newsgroup C. H. Lindsey [Page 11] News Article Architecture and Protocols November 2006 Published specification: [USEPRO] The content of the "application/news-groupinfo" body part is defined as: groupinfo-body = [ newsgroups-tag CRLF ] newsgroups-line CRLF newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP %x6E.65.77.73.67.72.6F.75.70.73 SP %x66.69.6C.65.3A ; case sensitive ; "For your newsgroups file:" newsgroups-line = newsgroup-name [ 1*HTAB newsgroup-description ] [ 1*WSP moderation-flag ] newsgroup-description = utext *( *WSP utext ) moderation-flag = %x28.4D.6F.64.65.72.61.74.65.64.29 ; case sensitive "(Moderated)" The MUST NOT contain any occurrence of the string "(Moderated)" within it. Although optional, the SHOULD be included until such time as this standard has been widely adopted, to ensure compatibility with present practice. Moderated newsgroups MUST be marked by appending the case sensitive text " (Moderated)" at the end. It is NOT recommended that the moderator's email address be included in the as has sometimes been done. NOTE: There is no provision for the use of charsets other than US-ASCII within a . Such a facility may be provided in a future extension to this standard. [That may seem harsh, but if we make any such provision now, it will make life more complicated and restrict our freedom when it comes to the proposed I18N extension. Therefore I resisted the temptation to include any charset parameter with this Media Type. Note that this also applies to the checkgroups message further on.] 4.4. Application/news-checkgroups The "application/news-checkgroups" Media Type is used in conjunction with the "checkgroups" control message (5.2.4). It MUST NOT be used except as a part of such control messages. The "application/news-checkgroups" body part contains a complete list of all the newsgroups in a (sub)hierarchy, their s and their moderation status. The MIME Media Type definition of "application/news-checkgroups" is: MIME type name: application MIME subtype name: news-checkgroups Required parameters: none C. H. Lindsey [Page 12] News Article Architecture and Protocols November 2006 Disposition: by default, inline Encoding considerations: "7bit" or "8bit" is sufficient and MUST be used to maintain compatibility. Security considerations: this type MUST NOT be used except as part of a checkgroups control message Published specification: [USEPRO] The content of the "application/news-checkgroups" body part is defined as: checkgroups-body = *( valid-group CRLF ) valid-group = newsgroups-line ; see 4.3 5. Control Messages The following sections document the control messages. "Message" is used herein as a synonym for "article" unless context indicates otherwise. Each comprises a , which indicates the action to be taken, and (s), which supply the details (see F- 3.2.3). The following sections contain syntactic definitions for the , s, and possibly the body, for each type of control message. The Newsgroups header field of each control message SHOULD include the (s) for the group(s) affected (i.e. groups to be created, modified or removed, or containing articles to be canceled). This is to ensure that the message propagates to all sites which receive (or would receive) that group(s). It MAY include other s so as to improve propagation (but this practice may cause the control message to propagate also to places where it is unwanted, or even cause it not to propagate where it should, so it should not be used without good reason). NOTE: Propagation is controlled by relaying agents, and it may be necessary for relaying agents to take special steps to ensure that control messages such as newgroup messages for not-yet- existent newsgroups are propagated correctly (see 6.3). The presence of a Subject header field whose content starts with the string "cmsg " followed by a was construed under [RFC 1036] as a request to perform that control action (even if no genuine Control header field was present). Indeed, some implementations went further and added the implied Control header field before injecting. Likewise, the presence of a ending in ".ctl" in the Newsgroups header field caused the Subject header field content (not starting with "cmsg" in this case) to be interpreted as a . All these practices, which have already largely fallen into disuse, are now declared to be Obsolete, and Subject header fields MUST NOT now be interpreted as s under any circumstances. C. H. Lindsey [Page 13] News Article Architecture and Protocols November 2006 [Possible addtional text:] In order to prevent continuing interpretation of Subject header fields in this way by existing agents, posting and injecting agents SHOULD detect and decline to post articles in which the Subject header field starts with the word "cmsg" and in which there is no Control header field. The descriptions below set out REQUIREMENTS to be followed by sites that receive control messages and choose to honour them. However, nothing in these descriptions should be taken as overriding the right of any such site, in accordance with its local policy, to refuse to honour any particular control message, or to refer it to an administrator for approval (either as a class or on a case-by-case basis). 5.1. Digital Signature of Header Fields It is most desirable that group control messages (5.2) in particular be authenticated by incorporating them within some digital signature scheme that encompasses other header fields closely associated with them (including at least Approved, Message-ID and Date). At the time of writing, this is usually done by means of a protocol known as "PGPverify" ([PGPVERIFY]), and continued usage of this is encouraged at least as an interim measure. However, PGPverify is not considered suitable for standardization in its present form, for various technical reasons. It is therefore expected that an early extension to this standard will provide a robust and general purpose digital authentication mechanism with applicability to all situations requiring protection against malicious use of, or interference with, header fields. That extension would also address other Netnews security issues. 5.2. Group Control Messages "Group control messages" are the sub-class of control messages that request some update to the configuration of the groups known to a storage agent, namely "newgroup", "rmgroup", "mvgroup" and "checkgroups", plus any others created by extensions to this standard. Group control messages that attempt to create groups with names that are deprecated or reserved according to F-3.1.4 MUST NOT be issued, except by prior agreement within some cooperating subnet. Moreover, sites receiving such control messages SHOULD check them for conformance before honouring them. All of the group control messages MUST have an Approved header field (F-3.2.1) which, in those hierarchies where appropriate administrative agencies exist (see 1.1), identifies the appropriate person or entity as authorized by those agencies. The authorized person or entity SHOULD adhere to any conventions and restrictions on the format of s established for those hierarchies C. H. Lindsey [Page 14] News Article Architecture and Protocols November 2006 [USEAGE]. 5.2.1. The 'newgroup' Control Message control-command =/ Newgroup-command Newgroup-command = "newgroup" Newgroup-arguments Newgroup-arguments = FWS newsgroup-name [ FWS newgroup-flag ] newgroup-flag = "moderated" The "newgroup" control message requests that the specified group be created or have its moderation status or changed. When the request is honoured, if the "moderated" is present then the status of the group SHOULD be marked as moderated, and vice versa. "Moderated" is the only such flag defined by this standard; other flags MAY be defined for use in cooperating subnets, but newgroup messages containing them MUST NOT be acted on outside of those subnets. NOTE: Specifically, some alternative flags such as "y" and "m", which are sent and recognized by some current software, are NOT part of this standard. Moreover, some existing implementations treat any flag other than "moderated" as indicating an unmoderated newsgroup. Both of these usages are contrary to this standard and control messages with such non-standard flags should be ignored. 5.2.1.1. The Body of the 'newgroup' Control Message The body of the newgroup message contains the following subparts, preferably in the order shown: 1. An "application/news-groupinfo" part (4.3) containing the name and (4.3) of the group. This part MUST be present and SHOULD be used to update any copy of the maintained by the storage agent. 2. Other parts containing useful information about the background of the newgroup message (typically of type "text/plain"). 3. Parts containing initial articles for the newsgroup. See section 5.2.1.2 for details. In the event that there is only the single (i.e. application/news- groupinfo) subpart present, it will suffice to include a "Content- Type: application/news-groupinfo" amongst the header fields of the control message. Otherwise, a "Content-Type: multipart/mixed" header field will be needed, and each separate part will then need its own Content-Type header field. 5.2.1.2. Initial Articles Some subparts of a "newgroup" or "mvgroup" control message MAY contain an initial set of articles to be posted to the affected newsgroup as soon as it has been created or modified. These parts are C. H. Lindsey [Page 15] News Article Architecture and Protocols November 2006 identified by having the Media Type "application/news-transmission", possibly with the parameter "usage=inject". The body of each such part should be a complete proto-article, ready for posting. This feature is intended for the posting of charters, initial FAQs and the like to the newly formed group. The Newsgroups header field of the proto-article MUST include the of the newly created or modified group. It MAY include other s. If the proto-article includes a Message-ID header field, the message identifier in it MUST be different from that of any existing article and from that of the control message as a whole. Alternatively such a message identifier MAY be derived by the injecting agent when the proto-article is posted. The proto-article SHOULD include the header field "Distribution: local". The proto-article SHOULD be injected at the storage agent that processes the control message AFTER the newsgroup in question has been created or modified. It MUST NOT be injected if the newsgroup is not, in fact, created (for whatever reason). It MUST NOT be submitted to any relaying agent for transmission beyond the storage agent(s) upon which the newsgroup creation has just been effected (in other words, it is to be treated as having a "Distribution: local" header field, whether such a field is actually present or not). NOTE: It is not precluded that the proto-article is itself a control message or other type of special article, to be activated only upon creation of the new newsgroup. However, except as might arise from that possibility, any "application/news-transmission" within some nested "multipart/*" structure within the proto-article is not to be activated. 5.2.1.3. Example A "newgroup" with its charter: From: "example.all Administrator" Newsgroups: example.admin.info,example.admin.announce Date: 27 Feb 2006 12:50:22 +0200 Subject: cmsg newgroup example.admin.info moderated Approved: admin@noc.example Control: newgroup example.admin.info moderated Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nxtprt" Content-Transfer-Encoding: 8bit This is a MIME control message. --nxtprt Content-Type: application/news-groupinfo For your newsgroups file: C. H. Lindsey [Page 16] News Article Architecture and Protocols November 2006 example.admin.info About the example.* groups (Moderated) --nxtprt Content-Type: application/news-transmission Newsgroups: example.admin.info From: "example.all Administrator" Subject: Charter for example.admin.info Message-ID: Distribution: local Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit The group example.admin.info contains regularly posted information on the example.* hierarchy. --nxtprt-- 5.2.2. The 'rmgroup' Control Message control-command =/ Rmgroup-command Rmgroup-command = "rmgroup" Rmgroup-arguments Rmgroup-arguments = FWS newsgroup-name The "rmgroup" control message requests that the specified group be removed from the list of valid groups. The Media Type of the body is unspecified; it MAY contain anything, usually an explanatory text. NOTE: It is entirely proper for a storage agent to retain the group until all the articles in it have expired, provided that it ceases to accept new articles. 5.2.2.1. Example From: "example.all Administrator" Newsgroups: example.admin.obsolete, example.admin.announce Date: 4 Apr 2006 22:04 -0900 (PST) Subject: cmsg rmgroup example.admin.obsolete Message-ID: Approved: admin@noc.example Control: rmgroup example.admin.obsolete The group example.admin.obsolete is obsolete. Please remove it from your system. 5.2.3. The 'mvgroup' Control Message control-command =/ Mvgroup-command Mvgroup-command = "mvgroup" Mvgroup-arguments Mvgroup-arguments = FWS newsgroup-name FWS newsgroup-name [ FWS newgroup-flag ] C. H. Lindsey [Page 17] News Article Architecture and Protocols November 2006 The "mvgroup" control message requests that the group specified by the first <(old-)newsgroup-name> be moved to that specified by the second <(new-)newsgroup-name>. Thus it is broadly equivalent to a "newgroup" control message for the second group followed by a "rmgroup" control message for the first group. The message body contains an "application/news-groupinfo" part (4.3) containing machine- and human-readable information about the new group, and possibly other subparts as for a "newgroup" control message. The information conveyed in the "application/news-groupinfo" body part, notably its (4.3), is applied to the new group. When this message is received, the new group is created (if it does not exist already) as for a "newgroup" control message, and SHOULD in any case be made moderated if a "moderated" is present, and vice versa. At the same time, arrangements SHOULD be made to remove the old group (as with a "rmgroup" control message), but only after a suitable overlap period to allow the network to adjust to the new arrangement. At the same time as a storage agent acts upon this message, all injecting agents associated with that storage agent SHOULD inhibit the posting of new articles to the old group (preferably with some indication to the poster that the new group should have been used). Relaying agents, however, MUST continue to propagate such articles during the overlap period. NOTE: It is to be expected that different storage agents will act on this message at different points of time, users of the old group will have to become accustomed to the new arrangement, and followups to already established threads will likely continue under the old group. Therefore, there needs to be an overlap period during which articles may continue to be accepted by relaying and storage agents in either group. This standard does not specify any standard period of overlap (though it would be expected to be expressed in days rather than in months). The inhibition of injection of new articles to the old group may seem draconian, but it is the surest way to prevent the changeover from dragging on indefinitely. Since the "mvgroup" control message is newly introduced in this standard and may not be widely implemented initially, it SHOULD be followed shortly afterwards by a corresponding "newgroup" control message; and again, after a reasonable overlap period, it MUST be followed by a "rmgroup" control message for the old group. In order to facilitate a smooth changeover, storage agents MAY arrange to service requests for access to the old group by providing access to the new group, which would then contain, or appear to contain, all articles posted to either group (including, ideally, the pre-changeover articles from the old one). Nevertheless, if this feature is implemented, the articles themselves, as supplied to reading agents, MUST NOT be altered in any way (and, in particular, C. H. Lindsey [Page 18] News Article Architecture and Protocols November 2006 their Newsgroups header fields MUST contain exactly those newsgroups present when they were injected). On the other hand, the Xref header field (F-3.2.14) MAY contain entries for either group (or even both). NOTE: Some storage agents that use an "active" file permit an entry of the form "oldgroup xxx yyy =newgroup", which enables any articles arriving for oldgroup to be diverted to newgroup, thus providing a simple implementation of this feature. However, it is known that not all current storage agents will find implementation so easy (especially in the short term) which is why it is not mandated by this standard. Nevertheless, its eventual implementation in all storage agents is to be considered highly desirable. On the other hand, it is recognized that this feature would likely not be implementable if the new group was already in existence with existing articles in it. This situation should not normally arise except when there is already some confusion as to which groups are, or are not, supposed to exist in that hierarchy. Note that the "mvgroup" control message is not really intended to be used for merging two existing groups. 5.2.3.1. Example From: "example.all Administrator" Newsgroups: example.oldgroup,example.newgroup,example.admin.announce Date: 30 Apr 2006 22:04 -0500 (EST) Subject: cmsg mvgroup example.oldgroup example.newgroup moderated Message-ID: Approved: admin@noc.example Control: mvgroup example.oldgroup example.newgroup moderated MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=nxt --nxt Content-Type: application/news-groupinfo For your newsgroups file: example.newgroup The new replacement group (Moderated) --nxt The moderated group example.oldgroup is replaced by example.newgroup. Please update your configuration, and please, if possible, arrange to file articles arriving for example.oldgroup as if they were in example.newgroup. --nxt Content-Type: application/news-transmission Newsgroups: example.admin.info From: "example.all Administrator" Subject: Charter for example.newgroup Message-ID: C. H. Lindsey [Page 19] News Article Architecture and Protocols November 2006 Distribution: local Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit This group (formerly known as example.oldgroup) is for the discussion of examples. --nxt-- 5.2.4. The 'checkgroups' Control Message The "checkgroups" control message contains a list of all the valid groups in a complete hierarchy. control-command =/ Checkgroup-command Checkgroup-command = "checkgroups" Checkgroup-arguments Checkgroup-arguments= [ chkscope ] [ chksernr ] chkscope = 1*( FWS ["!"] newsgroup-name ) chksernr = FWS "#" 1*DIGIT A "checkgroups" message applies to any (sub-)hierarchy with a prefix listed in the argument, provided that the rightmost matching in the list is not immediately preceded by a "!". If no argument is given, it applies to all hierarchies for which group statements appear in the body of the message. NOTE: Some existing software does not support the argument. Thus a "checkgroups" message SHOULD also contain the groups of other subhierarchies the sender is not responsible for. "New" software MUST ignore groups which do not fall within the argument of the "checkgroups" message. The argument is a serial number, which can be any positive integer (e.g. just numbered or the date in YYYYMMDD). It SHOULD increase by an arbitrary value with every change to the group list and MUST NOT ever decrease. NOTE: This was added to circumvent security problems in situations where the Date header field cannot be authenticated. Example: Control: checkgroups de !de.alt #248 which includes the whole of the 'de.*' hierarchy, with the exception of its 'de.alt.*' sub-hierarchy. The body of the message has the Media Type "application/news- checkgroups" (4.4). It asserts that the s it lists are the only newsgroups in the specified hierarchies. C. H. Lindsey [Page 20] News Article Architecture and Protocols November 2006 NOTE: The "checkgroups" message is intended to synchronize the list of newsgroups stored by a storage agent, and their s, with the lists stored by other storage agents throughout the network. However, it might be inadvisable for the storage agent actually to create or delete any newsgroups without first obtaining the approval of its administrators for such proposed actions. NOTE: The possibility of removing a complete hierarchy by means of an "invalidation" line beginning with a '!' in the checkgroups-body is no longer provided by this standard. The intent of the feature was widely misunderstood and it was misused more often than it was used correctly. The same effect, if required, can now be obtained by the use of an appropriate argument in conjunction with an empty . 5.3. Cancel The "cancel" message requests that a target article be "canceled", i.e. be withdrawn from circulation or access. control-command =/ Cancel-command Cancel-command = "cancel" Cancel-arguments Cancel-arguments = FWS msg-id [FWS] The argument identifies the article to be cancelled by its message identifier. The body SHOULD contain an indication of why the cancellation was requested. The "cancel" message SHOULD be posted to the same newsgroup(s), with the same distribution(s), as the article it is attempting to cancel. A storage agent that elects to honour a "cancel" message SHOULD make the article unavailable for relaying or storage (perhaps by deleting it completely). If the target article is unavailable, and the acceptability of the "cancel" message cannot be established without it, activation of the "cancel" message SHOULD be delayed until the target article has been seen. See also sections 6.3 and 6.4. NOTE: It is expected that the security extension envisaged in section 5.1 will make more detailed provisions for establishing whether honouring a particular "cancel" message is in order. In particular, it is likely that there will be provision for the digital signature of 3rd party cancels (i.e. those issued other than by the sender, the moderator, or the injector). NOTE: A cancel submitted by the poster for an article in a moderated group will be forwarded to the moderator of that group, and it is up to that moderator to act upon it (6.8). NOTE: The former requirement [RFC 1036] that the From and/or Sender header fields of the "cancel" message should match those of the original article has been removed from this standard, C. H. Lindsey [Page 21] News Article Architecture and Protocols November 2006 since it only encouraged cancel issuers to conceal their true identity, and it was not usually checked or enforced by canceling software. Therefore, both the From and/or Sender header fields and any Approved header field should now relate to the entity responsible for issuing the "cancel" message. 5.4. Ihave, sendme The "ihave" and "sendme" control messages implement a crude batched predecessor of the NNTP [RFC 3977] protocol. They are largely obsolete on the Internet, but still see use in conjunction with some transport protocols such as UUCP, especially for backup feeds that normally are active only when a primary feed path has failed. There is no requirement for relaying agents that do not support such transport protocols to implement them. NOTE: The ihave and sendme messages defined here have ABSOLUTELY NOTHING TO DO WITH NNTP, despite similarities of terminology. The two messages share the same syntax: control-command =/ Ihave-command Ihave-command = "ihave" Ihave-argument Ihave-argument = relayer-name control-command =/ Sendme-command Sendme-command = "sendme" Sendme-argument Sendme-argument = Ihave-argument relayer-name = path-identity ; see F-3.1.5 ihave-body = *( msg-id CRLF ) sendme-body = ihave-body The body of the message consists of a list of s, one per line. [RFC 1036] also permitted the list of s to appear in the or with the syntax Ihave-argument = [FWS] *( msg-id FWS ) [relayer-name] but this form SHOULD NOT now be used, though relaying agents MAY recognize and process it for backward compatibility. The "ihave" message states that the named relaying agent has received articles with the specified message identifiers, which may be of interest to the relaying agents receiving the ihave message. The "sendme" message requests that the agent receiving it send the articles having the specified message identifiers to the named relaying agent. Upon receipt of the sendme message, the receiving agent sends the article(s) requested, often (especially when the transport protocol is UUCP) in the form of one or more batches, each containing several articles. The usual form of a is defined by the following syntax (which is also used in the application/news transmission media type (4.1)). C. H. Lindsey [Page 22] News Article Architecture and Protocols November 2006 batch = 1*( batch-header article ) batch-header = "#!" SP rnews SP article-size CRLF rnews = %x72.6E.65.77.73 ; case sensitive "rnews" article-size = 1*DIGIT Thus a is a sequence of articles, each prefixed by a header line that includes its size. The is a decimal count of the octets in the article, counting each CRLF as one octet regardless of how it is actually represented. NOTE: Despite the similarity of this format to an executable UNIX script, it is EXTREMELY unwise to feed such a batch into a command interpreter in anticipation of it running a command named "rnews"; the security implications of so doing would be disastrous. These control messages are normally sent essentially as point-to- point messages, by using s in the Newsgroups header field of the form "to." followed by one (or possibly more) s in the form of a (see section F-3.1.4 which forbids "to" as the first of a ). The control message SHOULD then be delivered ONLY to the relaying agent(s) identified by that , and any relaying agent receiving such a message which includes its own MUST NOT propagate it further. Each pair of relaying agent(s) sending and receiving these messages MUST be immediate neighbours, exchanging news directly with each other. Each relaying agent advertises its new arrivals to the other using "ihave" messages, and each uses "sendme" messages to request the articles it lacks. To reduce overhead, ihave and sendme messages SHOULD be sent relatively infrequently and SHOULD contain reasonable numbers of message identifiers. If ihave and sendme are being used to implement a backup feed, it may be desirable to insert a delay between reception of an ihave and generation of a sendme, so that a slightly slow primary feed will not cause large numbers of articles to be requested unnecessarily via sendme. 5.5. Obsolete control messages. The following control messages (as described in Appendix A) are declared obsolete by this standard: sendsys version whogets senduuname 6. Duties of Various Agents The following section sets out the duties of various agents involved in the creation, relaying and storage of Netnews articles. Insofar as these duties are described as sequences of steps to be followed, it should be understood that it is the effect of these sequences that is C. H. Lindsey [Page 23] News Article Architecture and Protocols November 2006 important, and implementations may use any method that gives rise to that same effect. In this section, the word "verified", as applied to the source of some article, means that an agent processing that article has established, by some means, the identity of that source (which may be another agent or a poster). NOTE: In many implementations, a single agent may perform various combinations of the injecting, relaying and storage functions. Its duties are then the union of the various duties concerned. 6.1. General principles to be followed There are two important principles that news implementors (and administrators) need to keep in mind. The first is the well-known Internet Robustness Principle: Be liberal in what you accept, and conservative in what you send. However, in the case of news there is an even more important principle, derived from a much older code of practice, the Hippocratic Oath (we may thus call this the Hippocratic Principle): First, do no harm. It is VITAL to realize that decisions which might be merely suboptimal in a smaller context can become devastating mistakes when amplified by the actions of thousands of hosts within a few minutes. In the case of gateways, the primary corollary to this is: Cause no loops. 6.2. Duties of an Injecting Agent An Injecting Agent is responsible for taking a (proto-)article from a posting (or other) agent and either forwarding it to a moderator or injecting it into the relaying system for access by readers. As such, an injecting agent is considered responsible for ensuring that any article it injects conforms with the rules of [USEFOR]. It is also expected to bear some responsibility towards the rest of the network for the behaviour of its posters. In the normal course of events, an article that has already been injected into a Netnews network will never pass through another injecting agent. So, if an injecting agent receives an otherwise valid article that has already been injected (as evidenced by the presence of an Injection-Date header field, an Injection-Info header field, or more than one occurrence of the "POSTED" in a Path header field) it MAY choose to reject it, but otherwise SHOULD C. H. Lindsey [Page 24] News Article Architecture and Protocols November 2006 cause it to be relayed, as it stands, by a relaying agent (6.3). In exceptional circumstances (e.g. as part of some complex gatewaying process, or where a relaying agent considers it essential for fulfilling its responsibility towards the rest of the network) an already injected article MAY be "reinjected" into the network. This standard does not prescribe any such circumstance; rather this is a matter of policy to be determined by the administrators of each injecting agent, who have the responsibility to ensure that no harm arises. In all other circumstances, unintented reinjection is to be avoided (see 6.9). Nevertheless, in order to preserve the integrity of the network in these special cases, this standard does set out the correct way to reinject (see special provisions in 6.2.2 Steps 3, 7 and 9). It is usual for an injecting agent to be closely associated with a storage agent, thus giving it access to the list (6.4) showing the moderation status of the newsgroups it is likely to handle. In the event that it does not have such an associated storage agent, it MUST maintain that list itself. 6.2.1. Proto-articles A proto-article SHOULD NOT be propagated in that form to other than injecting agents. A proto-article has the same format as a normal article except that some of the following mandatory header fields MAY be omitted: Message-Id, Date, Path (and even From if the particular injecting agent can derive that information from other sources). However, if it is intended to offer the proto-article to two or more injecting agents in parallel, then it is only the Path header field that MAY be omitted. The header fields that can be omitted MUST NOT contain invalid values; they MUST either be correct or not present at all. [Maybe omit that last sentence.] NOTE: An article that is offered for reinjection has, by definition, already been injected once, and is not therefore to be considered as a proto-article. Hence a genuine proto-article will not contain any Injection-Date header field nor the "POSTED" anywhere in its Path header field, though that header field MAY contain s corresponding to machines traversed between the posting agent and the injecting agent proper. 6.2.2. Procedure to be followed by Injecting Agents An injecting agent receives (proto-)articles from posting and followup agents. It verifies them, adds header fields where required, and then either forwards them to a moderator or injects them by passing them to storage or relaying agents. It MUST NOT forward an already injected article to a moderator. C. H. Lindsey [Page 25] News Article Architecture and Protocols November 2006 An injecting agent processes articles as follows: 1. It MUST remove any Injection-Info header field already present (though it might be useful to copy it to a suitable "X-" header field). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace, or other non-standard tracing header field. 2. It SHOULD ensure that the article is from a verified source, and MAY reject articles in which header fields contain unverified email addresses, that is, addresses which are not known to be valid for the verified source, though it would be perverse to reject intentionally unverifiable addresses such as those ending in ".invalid" (6.5). 3. It SHOULD reject any article whose Date header field (F-3.1.1) is more than 24 hours into the future (and MAY use a margin less than that 24 hours). It MUST (except when reinjecting) reject any article with an Injection-Date header field already present (and SHOULD do likewise with any NNTP-Posting-Date header field). When reinjecting it MAY, in the absence of any Injection-Date header field, reject any article whose Date header field appears to be stale (e.g. more than 72 hours into the past). 4. It MUST reject any article that does not have the proper mandatory header fields for a proto-article or which contains any header field that does not have legal contents. It SHOULD reject any article which contains any header field deprecated for Netnews (e.g. as in [RFC 2298]). It SHOULD reject any article whose Newsgroups header field does not contain at least one for an existing group (as listed by its associated storage agent) and it MAY reject any which violates one of the restrictions in F-3.1.4 or which, although otherwise correct, violates a policy restriction established, for some (sub-)hierarchy, by an agency with the appropriate authority (1.2). Observe that crossposting to unknown newsgroups is not precluded provided at least one of those in the Newsgroups header field is listed. NOTE: This ability to reject s in breach of established policy does not extend to relaying agents, though it might be reasonable for posting agents to do it. 5. If the article is rejected (for reasons given above, or for other formatting errors or matters of site policy) the posting agent SHOULD be informed (such as via an NNTP 44x response code) that posting has failed and the article MUST NOT then be processed further. 6. The Message-ID, Date and From header fields (with appropriate contents) MUST be added when not already present. A User-Agent header field MAY be added (or an already present User-Agent header field MAY be augmented) so as to identify the software (e.g. "INN/1.7.2") used by the injecting agent. C. H. Lindsey [Page 26] News Article Architecture and Protocols November 2006 [I think that last sentence needs to go (in which case see consequential change in 7.3). Did we discuss this when we looked at User-Agent in USEFOR?] NOTE: The Message-ID, Date and From fields will already be present during reinjection. 7. The injecting agent MUST NOT alter the body of the article in any way (including any change of Content-Transfer-Encoding). It MAY (except when reinjecting) add other header fields not already provided by the poster, but SHOULD NOT alter, delete, or reorder any existing header field, with the specific exception of the "tracing" header field Injection-Info, which is to be removed as already mentioned. 8. If the Newsgroups header field contains one or more moderated groups and the article does NOT contain an Approved header field, the injecting agent MUST forward it to a moderator as specified in section 6.2.3 below. 9. Otherwise, a Path header field with a (F-3.1.5) MUST be correctly added if not already present. During reinjection, the existing Path header field SHOULD be retained. 10.It MUST then prepend the of the injecting agent, followed by '!.' and the "POSTED", and then a further "!", to the content of the Path header field; this header field SHOULD then be folded if it would otherwise result in a header line of excessive length. NOTE: This could result in more that one "POSTED" in the case of reinjection. 11.An Injection-Info header field (F-3.2.8) SHOULD be added, identifying the verified source of the article and possibly an address for mailing complaints to. Each injecting agent SHOULD use a consistent form of the Injection-Info header field for all articles emanating from the same or similar origins. NOTE: The step above is the only place in which an Injection- Info header field is to be created. It follows that this header field MUST NOT be created, replaced, changed or deleted by any other agent (except during reinjection, in which case it will always relate to the latest injection and is, to that extent, regarded as a variant header field). 12.The injecting agent MUST then add an Injection-Date header field (F-3.2.7) if one is not already present, but it MUST NOT alter, or delete, an already present Injection-Date header field (and likewise SHOULD NOT alter, or delete, an already present NNTP- Posting-Date header field). Finally, it forwards the article to one or more relaying or storage agents, and the injection process is to be considered complete. C. H. Lindsey [Page 27] News Article Architecture and Protocols November 2006 NOTE: The step above is the only place where an Injection-Date header field is to be created It follows that it MUST NOT subsequently be replaced, changed or deleted by any other agent, even during reinjection. 6.2.3. Procedure for Forwarding to a Moderator An injecting agent forwards an article to a moderator as follows: 1. It MUST forward it to the moderator of the first (leftmost) moderated group listed in the Newsgroups header field, customarily via email, (see 6.8 for how that moderator may forward it to further moderators). There are two possibilities for doing this: (a) The complete article is encapsulated (header fields and all) within the email, preferably using the Content-Type "application/news-transmission" (4.1) with any usage parameter set to "moderate". Moreover, there SHOULD NOT be more than one encapsulated article within the one email. This method has the advantage of removing any possible conflict between Netnews and Email header fields, or of changes to those fields during transport through email. (b) The article is sent as an email as it stands, with the addition of such extra header fields (e.g. a To header field) as are necessary for an email. The existing Message-ID header field SHOULD be retained. Although both of these methods have seen use in the past, the preponderance of current usage on Usenet has been for method (b) and many moderators are ill-prepared to deal with method (a). Therefore, method (a) SHOULD NOT be used until such time as the majority of moderators are able to accept it. 2. This standard does not prescribe how the email address of the moderator is to be determined, that being a matter of policy to be arranged by the agency responsible for the oversight of each hierarchy. Nevertheless, there do exist various agents worldwide which provide the service of forwarding to moderators, and the address to use with them is obtained as follows: (a) Each '.' in the is replaced with a '-'. (b) The result of these operations is used as the of the of the agent. For example, articles intended for "news.announce.important" would be emailed to "news- announce-important@forwardingagent.example". 6.3. Duties of a Relaying Agent A Relaying Agent accepts injected articles from injecting and other relaying agents and passes them on to relaying or storage agents according to mutually agreed policy. Relaying agents SHOULD accept articles ONLY from verified sources. C. H. Lindsey [Page 28] News Article Architecture and Protocols November 2006 An article SHOULD NOT be relayed unless the sending agent has been configured to supply and the receiving agent to receive at least one of the s in its Newsgroups header field and at least one of the s in its Distribution header field, if any. Exceptionally, ALL relaying agents are deemed willing to supply or accept the "world", and NO relaying agent should supply or accept the "local". However, if the particular implementation does not relay non-existent newsgroups, even when included in the Newsgroups header field and implied (e.g. by some "wild card" notation) in the configuration tables, then the agent MUST examine all group control messages (5.2) in order to ensure that relaying of those messages proceeds normally. NOTE: Although it would seem redundant to filter out unwanted distributions at both ends of a relaying link (and it is clearly more efficient to do so at the sending end), many sending sites have been reluctant, historically speaking, to apply such filters (except to ensure that distributions local to their own site or cooperating subnet did not escape); moreover they tended to configure their filters on an "all but those listed" basis, so that new and hitherto unheard of distributions would not be caught. Indeed many "hub" sites actually wanted to receive all possible distributions so that they could feed on to their clients in all possible geographical (or organizational) regions. Therefore, it is desirable to provide facilities for rejecting unwanted distributions at the receiving end. Indeed, it may be simpler to do so locally than to inform each sending site of what is required, especially in the case of specialized distributions (for example for control messages, such as cancels from certain issuers) which might need to be added at short notice. A similar possibility for reading agents to filter distributions is also suggested in [USEAGE]) for the same reason. In order to avoid unnecessary relaying, an article SHOULD NOT be relayed if the of the receiving agent (or some known alias thereof) appears as a in its Path header field. But note that the (which follows the last "!") is not a , although not all current implementations observe this distinction. For this to work, each relaying agent needs to insert it own (chosen according to 2.3) into the Path header field. It MAY insert more than one for itself (in which case the leftmost should be the one by which it is known to its immediate neighbours), but where an article passes through several relaying agents at the same site it MAY omit the s for some of them (but NOT the one which finally relays it to an outside site). C. H. Lindsey [Page 29] News Article Architecture and Protocols November 2006 It MAY (and usually SHOULD) also add a giving additional information concerning the route taken by the article through the network. A consists of either the special "!" (which effectively replaces the standard delimiter "!" by "!!"), or it is composed from a usually followed by a . The following are the only s defined by this standard: o "POSTED" (already introduced in Step 10 of 6.2.2), which is never followed by a ; o "SEEN", whose following indicates the verified identity (see 6) of the agent from which the article was received (but makes no claim as to whether it matched the indicates the verified identity of the agent from which the article was received and asserts that it could not be reconciled with the (supposedly) inserted by that agent. Other s beginning with "X" MAY be used by a relaying agent to make some assertion not envisaged here, but other (non-"X") s MUST NOT be used unless defined by some extension to this standard. NOTE: The "MATCH", which might have indicated the verified identity of the source agent with an assertion that it agreed with the inserted by that agent, has NOT been provided, since the special conveys exactly that meaning for this commonly occurring case. NOTE: Whilst s are case insensitive, it is intended that they should normally be rendered in uppercase. A relaying agent processes articles as follows: 1. It MUST/SHOULD establish the verified identity of the source of the article and compare it with the leftmost of the existing Path header field's content. Except possibly when relaying to other hosts on the same site, It then MUST or SHOULD, as indicated, prepend to that content (from left to right) the following: o (MUST) its own ; o (MUST) a "!" delimiter; o (MUST/SHOULD) if the verified and existing identities match, a (effectively converting the "!" delimiter into "!!"); o (MUST/SHOULD) alternatively, where the identities do not match (or have not been determied to match), a ".", the "MISMATCH" (or "SEEN"), another ".", a indicating the verified identity, and finally a further "!"; o possibly further s etc. as above, identifying itself. This header field SHOULD then be folded if it would otherwise result in a header line of excessive length. C. H. Lindsey [Page 30] News Article Architecture and Protocols November 2006 [The "MUST/SHOULD"s above were all "MUST" in the previous drafts. Discussion is needed to resolve this.] NOTE: Since each agent at one site can be assumed to be aware of the identity of the others (and of itself), it would be most unusual for their s to be separated other than by "!!". Thus the presence of a single "!", unless followed by a "." and some , can be taken as signifying an agent that has not yet been upgraded to conform to this standard. NOTE: Whilst the presence of a "MISMATCH" would normally indicate that the existing Path was bogus in some sense, it could also indicate that the ralaying agent was improperly configured to recognise the identities or aliases used by its neighbours. Administators of relaying agents should therefore periodically monitor the being inserted so as to avoid this. NOTE: In order to prevent overloading, relaying agents should not routinely query an external entity (such as a DNS-server) in order to determine a verified identity (though a local cache of the required information might usefully be consulted). 2. It MUST examine the Injection-Date header field (or, if that is absent, the Date header field) and reject the article as stale (F-3.2.7) if that predates the earliest articles of which it normally keeps record, or if it is more than 24 hours into the future (the margin MAY be less than that 24 hours). 3. It SHOULD reject any article that does not include all the mandatory header fields (section F-3.1). 4. It MAY reject any article whose header fields do not have legal contents. 5. It SHOULD reject any article that has already been sent to it (a database of message identifiers of recent messages is usually kept and matched against). NOTE: Even though commonly derived from the domain name of the originating site (and domain names are case-insensitive), a MUST NOT be altered in any way during transport, or when copied (as when forming a References header field), and thus a simple (case-sensitive) comparison of octets will always suffice to recognize that same message identifier wherever it subsequently reappears. NOTE: These requirements are to be contrasted with those of the un-normalized msg-ids defined by [RFC 2822], which may perfectly legitimately become normalized (or vice versa) during transport or copying in email systems. C. H. Lindsey [Page 31] News Article Architecture and Protocols November 2006 6. It SHOULD reject any article that matches an already received cancel message (or an equivalent Supersedes header field) issued by its poster or by some other trusted entity. 7. It MAY reject any article without an Approved header field posted to newsgroups known to be moderated (this practice is strongly recommended, but the information necessary to do so may not be available to all agents). 8. It MAY delete any Xref header field that is present. 9. Finally, it passes the articles on to neighbouring relaying and storage agents. If the article is rejected as being invalid, unwanted or unacceptable due to site policy, the agent that passed the article to the relaying agent SHOULD be informed (such as via an NNTP 43x response code) that relaying failed. In order to prevent a large number of error messages being sent to one location, relaying agents MUST NOT inform any other external entity that an article was not relayed UNLESS that external entity has explicitly requested that it be informed of such errors. Relaying agents MUST NOT alter, delete or rearrange any part of an article except for header fields designated as variant (2.4). In particular o they MUST NOT create or augment a User-Agent header field in order to identify themselves; o they MUST NOT rewrite the Newsgroups header field in any way, even if some supposedly non-existent newsgroup is included; o they MUST NOT refold any header field (i.e. they must pass on the folding as received); o they MUST NOT alter the Date header field or the Injection-Date header field; o they MUST NOT delete any unrecognized header field whose field name is syntactically correct (whether or not it is registered with IANA [RFC 3864]); o they MUST NOT change the Content-Transfer-Encoding of the body or any body part; o they MUST transmit lines of arbitrary length and articles of arbitrary size. 6.3.1. Path Header Field Example Path: bar.isp.example!.SEEN.news3.foo.isp.example!foo.isp.example !!foo-server!.MISMATCH.2001:DB8:0:0:8:800:200C:417A !dubious.site.example!!old.site.example!barbaz!!baz.isp.example !.POSTED!dialup123.baz.isp.example!not-for-mail NOTE: That article was injected into the news stream by baz.isp.example, as indicated by the "POSTED" (complaints may be addressed to abuse@baz.isp.example). The injector has chosen to record that it got it from dialup123.baz.isp.example. "not-for-mail" is a dummy , though sometimes a real userid is put there. The article was relayed, perhaps by UUCP, to the machine known, at least to old.site.example, as "barbaz". Barbaz relayed it to old.site.example, which does not yet conform to this standard (hence the single '!' delimiter). So one cannot be sure that it really came from barbaz. Old.site.example relayed it to a site claiming to be dubious.site.example, and claiming (by using the '!!' delimiter) to have verified that it came from old.site.example. Dubious.site.example relayed it to "foo-server" which, not being convinced that it truly came from dubious.site.example, and knowing that it in fact arrived from a host with the IPv6address [2001:DB8:0:0:8:800:200C:417A], inserted the "!.MISMATCH.2001:DB8:0:0:8:800:200C:417A" (that is not to say that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6 address for dubious-site.example, but simply that that connection could not be substantiated by foo-server). "foo-server" is a locally significant name within the complex site of many machines run by foo.isp.example, so the latter should have no problem recognizing foo-server and using a '!!' delimiter. It was not strictly necessary to insert the "foo-server" as well as "foo.isp.example" (but maybe some site elsewhere had some reason to test for it). [Please could Richard Clayton provide a more plausible reason why "foo- server" might be a here?] It then went to bar.isp.example which determined (by reverse DNS) that it had come from news3.foo.isp.example, but had taken no steps to check whether that was a known alias for foo.isp.example (which it probably was). Strictly, it SHOULD have made such a check, and then inserted either a "!!" or a "!MISMATCH..." according to the result. Presumably bar.isp.example then delivered the article to its direct clients. It appears that foo.isp.example, foo-server and baz.isp.example decided to fold the line, on the grounds that it seemed to be getting a little too long. Note that folding and whitespace is permitted before (but not after) any "!" (but not within a "!!"); hence continuation lines will always start with either "!" or "!!". 6.4. Duties of a Serving Agent A Serving Agent takes an article from a relaying or injecting agent and files it in a "news database". It also provides an interface for reading agents to access the news database. This database is normally indexed by newsgroup with articles in each newsgroup identified by an C. H. Lindsey [Page 33] News Article Architecture and Protocols November 2006 (usually in the form of a decimal number - see F- 3.2.14). A storage agent MUST maintain a list of the newsgroups it stores in its news database showing the moderation status of each one (see 5.2.1), and SHOULD include in that list all groups likely to be crossposted to from those groups (e.g. all other groups in the same hierarchy(ies)). NOTE: Since control messages are often of interest, but should not be displayed as normal articles in regular newsgroups, it is common for storage agents to make them available in a pseudo- newsgroup named "control" or in a pseudo-newsgroup in a sub- hierarchy under "control." (e.g. "control.cancel"). A storage agent MAY decline to accept an article if the Path header field contains some whose articles the storage agent does not want, as a matter of local policy. NOTE: This last facility is sometimes used to detect and decline control messages (notably cancel messages) which have been deliberately seeded with a to be "aliased out" by sites not wishing to act upon them. [INN at least does this. It might be argued that it is not necessary to mention it here.] A storage agent processes articles as follows: 1. It MUST/SHOULD establish the verified identity of the source of the article and modify the Path header field as for a relaying agent (6.3). 2. It MUST examine the Injection-Date header field (or, if that is absent, the Date header field) and reject the article as stale (F-3.2.7) if that predates the earliest articles of which it normally keeps record, or if it is more than 24 hours into the future (the margin MAY be less than that 24 hours). 3. It MUST reject any article that does not include all the mandatory header fields (section F-3.1), or which contains any header field that does not have legal contents. 4. It SHOULD reject any article that has already been sent to it (a database of message identifiers of recent articles is usually kept and matched against). 5. It SHOULD reject any article that matches an already received cancel message (or an equivalent Supersedes header field) issued by its poster or by some other trusted entity. Likewise, a newly received cancel message (or equivalent Supersedes) should cause immediate deletion (or deactivation) of the canceled article. C. H. Lindsey [Page 34] News Article Architecture and Protocols November 2006 NOTE: An article with a Supersedes header field is itself stored normally. 6. It MUST reject any article without an Approved header field posted to any newsgroup listed as moderated. 7. It MUST (exept when specially configured to preserve the s set by the sending site) remove any Xref header field (F-3.2.14) from each article. It then MAY (and usually will) generate a fresh Xref header field. 8. Finally, it stores the article in its news database. Serving agents MUST NOT create new newsgroups simply because an unrecognized occurs in a Newsgroups header field (see 5.2.1 for the correct method of newsgroup creation). Serving agents MUST NOT alter, delete or rearrange any part of an article in any other way. The list of particular cases given for relaying agents (6.3) applies here also. 6.5. Duties of a Posting Agent A Posting Agent is used to assist the poster in creating a valid proto-article and forwarding it to an injecting agent. Postings agents SHOULD ensure that proto-articles they create are valid according to [USEFOR] and other applicable policies. In particular, they MUST NOT create any Injection-Date or Injection-Info header field. Contrary to [RFC 2822], which implies that the mailbox(es) in the From header field should be that of the poster(s), a poster who does not, for whatever reason, wish to use his own mailbox MAY use any mailbox ending in the top level domain ".invalid" [RFC 2606]. Posting agents meant for use by ordinary posters SHOULD reject any attempt to post an article which cancels or Supersedes another article of which the poster is not the author or sender. 6.6. Duties of a Followup Agent A Followup Agent is a special case of a posting agent, and as such is bound by all the posting agent's requirements. Followup agents MUST create valid followups and are subject to special requirements involving the Newsgroups, Subject, Distribution and References header fields. Wherever in the following it is stated that, "by default", a header field is to be "inherited" from one of those header fields in the precursor, it means that its initial (semantic) content is to be a copy of the content of that precursor header field. However, posters MAY then override that default before posting if they so wish. C. H. Lindsey [Page 35] News Article Architecture and Protocols November 2006 NOTE: The Keywords header field is not inheritable, though some older user agents treated it as such. 1. The Newsgroups header field (F-3.1.4) SHOULD by default be inherited from the precursor's Followup-To header field if present, and otherwise from the precursor's Newsgroups header field. However, if the content of that Followup-To header field consists of "poster" (and the user does not override it), then the followup MUST NOT be posted but, rather, is to be emailed to the precursor's poster. 2. The Subject header field SHOULD by default be inherited from that of the precursor. The case sensitive string "Re: " MAY be prepended to the content of its Subject header field, unless it already begins with that string. 3. The Distribution header field (F-3.2.4) SHOULD by default be inherited from the precursor's Distribution header field, if any. 4. The followup MUST (in accordance with the definition of that term - F-1.5) have a References header field referring to its precursor, constructed in accordance with section 6.6.1 below. NOTE: That "MUST" is to be contrasted with the weaker recomendation using "SHOULD" applied, in [RFC 2822], to the generation of "replies" in email. Moreover, in Netnews, there is no expectation of any In-Reply-To header field in a followup. 6.6.1. Construction of the References header field The following procedure is to be used whenever some previous article (the "parent") is to be referred to in the References header field (F-3.2.10) of a new article, whether in the course of generating a followup or for some other reason (e.g. the later parts of a multipart posting such as a FAQ, or the later parts of a message/partial as suggested in [RFC 2046]). The (semantic) content (2.1) of the new article's References header field consists of the content of the Message-ID header field of the parent preceded, if the parent had a References header field, by the content of that References header field and a SP (subject to trimming as described below). If the resulting References header field would, after unfolding, exceed 998 characters in length (including its field name but not the final CRLF), it MUST be trimmed (and otherwise it MAY be trimmed). Trimming involves removing any number of message identifiers from its content, except that the first message identifier and the last two MUST NOT be removed. NOTE: There is no provision in this standard for an article to have more than one parent. The essential property of the References header field, guaranteed by the procedure above and to be preserved in any future extension, is that no article can C. H. Lindsey [Page 36] News Article Architecture and Protocols November 2006 ever precede one of its own parents. 6.7. Duties of a Reading Agent A reading agent downloads articles from a storage agent, as directed by the reader, and displays them to the reader (or processes them in some other manner). It SHOULD also have the capability to show the raw article exactly as received. It MAY present lists of articles available for display, and MAY structure those lists so as to show the relationships between the articles, as determined by the References, Subject, Date and other header fields (see [USEAGE] for some usual methods of doing this). [This whole section may yet get omitted] 6.8. Duties of a Moderator A Moderator receives news articles, customarily by email, decides whether to approve them and, if so, either injects them into the news stream or forwards them to further moderators. Articles will be received by the moderator either encapsulated as an object of Content-Type application/news-transmission (or possibly encapsulated but without an explicit Content-Type header field), or else directly as an email already containing all the header fields appropriate for a Netnews article (see 6.2.2). Moderators SHOULD be prepared to accept articles in either format. A moderator processes an article, as submitted to any newsgroup that he moderates, as follows: 1. He decides, on the basis of whatever moderation policy applies to his group, whether to approve or reject the article. He MAY do this manually, or else partially or wholly with the aid of appropriate software for whose operation he is then responsible. If the article is a cancel nessage (5.3) issued by the poster of an earlier article, then he is expected to cancel that earlier article (in which case there is no more to be done). He MAY modify the article if that is in accordance with the applicable moderation policy (and in particular he MAY remove redundant header fields and add Comments and other informational header fields). He also needs to be aware if any change he makes to the article will invalidate some authentication check provided by the poster or by an earlier moderator. If the article is rejected, then it normally fails for all the newsgroups for which it was intended. If it is approved, the moderator proceeds with the following steps. 2. If the Newsgroups header field contains further moderated newsgroups for which approval has not already been given, he adds an indication (identifying both himself and the name of the group) that he approves the article, and then forwards it to the moderator of the leftmost unapproved group (which, if this C. H. Lindsey [Page 37] News Article Architecture and Protocols November 2006 standard has been followed correctly, will generally be the next moderated group to the right of his own). There are two ways to do this: (a) He emails it to the submission address of the next moderator (see section 6.2.2 for the proper method of doing this), or (b) he rotates the s in the Newsgroups header field to the left so that the targeted group is the leftmost moderated group in that field, and injects it again (thus causing the injecting agent to forward it to the correct moderator). However, he MUST first ensure that the article contains no Approved header field. NOTE: This standard does not prescribe how a moderator's approval is to be indicated (though a future standard may do so). Possible methods include adding an Approved header field (or a similar but differently named header field if method (b) is being used) listing all the approvals made so far, or adding a separate header field for each individual approval (the header field X-Auth is sometimes used for this purpose). The approval may also be confirmed with some form of digital signature (5.1). 3. If the Newsgroups header field contains no further unapproved moderated groups, he adds an Approved header field (F-3.2.1) identifying himself and, insofar as is possible, all the other moderators who have approved the article. He thus assumes responsibility for having ensured that the article was approved by the moderators of all the moderated groups involved. 4. The Date header field SHOULD be retained. Any Injection-Date header field already present (though there should be none) MUST be removed. Exceptionally, if it is known that the injecting agent does not yet support the Injection-Date header field and the Date header field appears to be stale (F-3.2.7) for reasons understood by the moderator (e.g. delays in the moderation process) he MAY substitute the current date. The Message-ID header field SHOULD also be retained unless it is obviously non-compliant with [USEFOR]. NOTE: A created by a conforming posting or injecting agent, or even by a mail user agent conforming to [RFC 2822], may reasonably be supposed to be conformant (and will, in any case, be caught by the injecting agent if it is not). 5. Any variant header fields (2.4) MUST be removed, except that a Path header field MAY be truncated to only those entries following its "POSTED" . Any Injection-Info header field (F- 3.2.8) SHOULD be removed (and if not, the injecting agent will do so, as required in 6.2.2). 6. He then causes the article to be injected, having first observed all the duties of a posting agent. C. H. Lindsey [Page 38] News Article Architecture and Protocols November 2006 NOTE: This standard does not prescribe how the moderator or moderation policy for each newsgroup is established; rather it assumes that whatever agencies are responsible for the relevant network or hierarchy (1.1) will have made appropriate arrangements in that regard. 6.9. Duties of a Gateway A Gateway transforms an article into the native message format of another medium, or translates the messages of another medium into news articles. Encapsulation of a news article into a message of MIME type application/news-transmission, or the subsequent undoing of that encapsulation, is not gatewaying, since it involves no transformation of the article. There are two basic types of gateway, the Outgoing Gateway that transforms a news article into a different type of message, and the Incoming Gateway that transforms a message from another medium into a news article and injects it into a news system. These are handled separately below. The primary diktat for a gateway is: Above all, prevent loops. Transformation of an article into another medium stands a very high chance of discarding or interfering with the protection inherent in the news system against duplicate articles. The most common problem caused by gateways is "spews", gateway loops that cause previously posted articles to be reinjected repeatedly into Usenet. To prevent this, a gateway MUST take precautions against loops, as detailed below. If bidirectional gatewaying (both an incoming and an outgoing gateway) is being set up between Netnews and some other medium, the incoming and outgoing gateways SHOULD be coordinated to avoid unintended reinjection of gated articles. Circular gatewaying (gatewaying a message into another medium and then back into Netnews) SHOULD NOT be done; encapsulation of the article SHOULD be used instead where this is necessary. A second general principal of gatewaying is that the transformations applied to the message SHOULD be as minimal as possible while still accomplishing the gatewaying. Every change made by a gateway potentially breaks a property of one of the media or loses information, and therefore only those transformations made necessary by the differences between the media should be applied. It is worth noting that safe bidirectional gatewaying between a mailing list and a newsgroup is far easier if the newsgroup is moderated. Posts to the moderated group and submissions to the mailing list can then go through a single point that does the necessary gatewaying and then sends the message out to both the newsgroup and the mailing list at the same time, eliminating most of C. H. Lindsey [Page 39] News Article Architecture and Protocols November 2006 the possibility of loops. Bidirectional gatewaying between a mailing list and an unmoderated newsgroup, in contrast, is difficult to do correctly and is far more fragile. Newsgroups intended to be bidirectionally gated to a mailing list SHOULD therefore be moderated where possible, even if the moderator is a simple gateway and injecting agent that correctly handles crossposting to other moderated groups and otherwise passes all traffic. 6.9.1. Duties of an Outgoing Gateway From the perspective of Netnews, an outgoing gateway is just a special type of reading agent. The exact nature of what the outgoing gateway will need to do to articles depends on the medium to which the articles are being gated. The operation of the outgoing gateway is subject to additional constraints due to the possibility of one or more corresponding incoming gateways back from that medium to Netnews, since this opens the possibility of loops. In general, the following practices are recommended for all outgoing gateways, regardless of whether there is known to be a related incoming gateway, both as a precautionary measure and as a guideline to quality of implementation. 1. The message identifier of the news article should be preserved if at all possible, preferably as or within the corresponding unique identifier of the other medium, but if not at least as a comment in the message. This helps greatly with preventing loops. 2. The Date and Injection-Date of the news article should also be preserved if possible, for similar reasons. 3. The message should be tagged in some way so as to prevent its reinjection into Netnews. This may be impossible to do without knowledge of potential incoming gateways, but it is better to try to provide some indication even if not successful; at the least, a human-readable indication that the article should not be gated back to Netnews can help locate a human problem. 4. Netnews control messages should not be gated to another medium unless they would somehow be meaningful in that medium. 5. Changes MAY be made to the Content-Transfer-Encoding of some or all parts of the body, and even to the charsets specified in s or in Content-Type header fields, but such changes SHOULD NOT be made unless absolutely necessary. 6.9.2. Duties of an Incoming Gateway The incoming gateway has the serious responsibility of ensuring that all of the requirements of this standard are met by the articles that it forms. In addition to its special duties as a gateway, it bears all of the duties and responsibilities of an injecting agent as well, C. H. Lindsey [Page 40] News Article Architecture and Protocols November 2006 and additionally has the same responsibility of a relaying agent to reject articles that it has already gatewayed. An incoming gateway MUST NOT gate the same message twice. It may not be possible to ensure this in the face of mangling or modification of the message, but at the very least a gateway, when given a copy of a message it has already gated identical except for trace header fields (like Received in Email or Path in Netnews) MUST NOT gate the message again. An incoming gateway SHOULD take precautions against having this rule bypassed by modifications of the message that can be anticipated. News articles prepared by gateways MUST be legal news articles. In particular, they MUST include all of the mandatory header fields, MUST fully conform to the restrictions on those fields, and SHOULD exclude any deprecated header fields (e.g. as in [RFC 2298]). This often requires that a gateway function not only as a relaying agent, but also partly as a posting agent, aiding in the synthesis of a conforming article from non-conforming input. Incoming gateways MUST NOT pass control messages (articles containing a Control or Supersedes header field) without removing or renaming that header field. Gateways MAY, however, generate their own cancel messages, under the general allowance for injecting agents to cancel their own messages ([USEAGE]). If a gateway receives a message that it can determine is a valid equivalent of a cancel message in the medium it is gatewaying, it SHOULD discard that message without gatewaying it, generate a corresponding cancel message of its own, and inject that cancel message. Incoming gateways MUST NOT inject control messages other than cancels. Encapsulation SHOULD be used instead of gatewaying, when direct posting is not possible or desirable. NOTE: It is not unheard of for mail-to-news gateways to be used to post control messages, but encapsulation should be used for these cases instead. Gateways by their very nature are particularly prone to loops. Spews of normal articles are bad enough; spews of control messages with special significance to the news system, possibly resulting in high processing load or even email sent for every message received, are catastrophic. It is far preferable to construct a system specifically for posting control messages that can do appropriate consistency checks and authentication of the originator of the message. If there is a message identifier that fills a role similar to that of the Message-ID header field in news, it SHOULD be used in the formation of the message identifier of the news article, perhaps with transformations required to meet the uniqueness requirement of Netnews and with the removal of any comments so as to comply with the syntax in section F-3.1.3. Such transformations SHOULD be designed so that two messages with the same identifier generate the same Message-ID header field. C. H. Lindsey [Page 41] News Article Architecture and Protocols November 2006 NOTE: Message identifiers play a central role in the prevention of duplicates, and their correct use by gateways will do much to prevent loops. Netnews does, however, require that message identifiers be unique, and therefore message identifiers from other media may not be suitable for use without modification. A balance must be struck by the gateway between preserving information used to prevent loops and generating unique message identifiers. Exceptionally, if there are multiple incoming gateways for a particular set of messages, each to a different newsgroup(s), each one SHOULD generate a message identifier unique to that gateway. Each incoming gateway nonetheless MUST ensure that it does not gate the same message twice. NOTE: Consider the example of two gateways of a given mailing list into the world-wide Usenet newsgroups, both of which preserve the email message identifier. Each newsgroup may then receive a portion of the messages (different sites seeing different portions). In these cases, where there is no one "official" gateway, some other method of generating message identifiers has to be used to avoid collisions. It would obviously be preferable for there to be only one gateway which crossposts, but this may not be possible to coordinate. If no date information is available, the gateway MAY supply a Date header field with the gateway's current date. If no injection-date information is available, the gateway MUST supply an Injection-Date header field with whatever date information is available, and otherwise with the gateway's current date. If only partial information is available (e.g. date but not time), this SHOULD be fleshed out to a full Date and/or Injection-Date header field by adding default values rather than discarding this information. Only in very exceptional circumstances should Date information be discarded, as it plays an important role in preventing reinjection of old messages. An incoming gateway MUST add a Sender header field to the news article it forms containing the of the administrator of the gateway. Problems with the gateway may be reported to this . The portion of this SHOULD indicate that the entity responsible for injection of the message is a gateway. If the original message already had a Sender header field, it SHOULD be renamed so that its contents can be preserved. 6.9.3. Example To illustrate the type of precautions that should be taken against loops, here is an example of the measures taken by one particular combination of mail-to-news and news-to-mail gateways at Stanford University designed to handle bidirectional gatewaying between mailing lists and unmoderated groups. C. H. Lindsey [Page 42] News Article Architecture and Protocols November 2006 1. The news-to-mail gateway preserves the message identifier of the news article in the generated email message. The mail-to-news gateway likewise preserves the email message identifier provided that it is syntactically valid for Netnews. This allows the news system's built-in suppression of duplicates to serve as the first line of defense against loops. 2. The news-to-mail gateway adds an X-Gateway header field to all messages it generates. The mail-to-news gateway discards any incoming messages containing this header field. This is robust against mailing list managers that replace the message identifier, and against any number of email hops, provided that the other message header fields are preserved. 3. The mail-to-news gateway prepends the host name from which it received the email message to the content of the Path header field. The news-to-mail gateway refuses to gateway any message that contains the list server name in its Path header field. This is robust against any amount of munging of the message header fields by the mailing list, provided that the email only goes through one hop. 4. The mail-to-news gateway is designed never to generate bounces to the envelope sender. Instead, articles that are rejected by the news server (for reasons not warranting silent discarding of the message) result in a bounce message sent to an errors address known not to forward to any mailing lists, so that they can be handled by the news administrators. These precautions have proven effective in practice at preventing loops for this particular application (bidirectional gatewaying between mailing lists and locally distributed newsgroups where both gateways can be designed together). General gatewaying to world-wide newsgroups poses additional difficulties; one must be very wary of strange configurations, such as a newsgroup gated to a mailing list which is in turn gated to a different newsgroup. 7. Security and Related Considerations There is no security. Don't fool yourself. Usenet is a prime example of an Internet Adhocratic-Anarchy; that is, an environment in which trust forms the basis of all agreements. It works. See also F-5 for further security considerations related to the format of articles. 7.1. Leakage Articles which are intended to have restricted distribution are dependent on the goodwill of every site receiving them. The "Archive: no" header field (F-3.2.2) is available as a signal to automated archivers not to file an article, but that cannot be guaranteed. C. H. Lindsey [Page 43] News Article Architecture and Protocols November 2006 The Distribution header field makes provision for articles which should not be propagated beyond a cooperating subnet. The key security word here is "cooperating". When a machine is not configured properly, it may become uncooperative and tend to distribute all articles. The flooding algorithm is extremely good at finding any path by which articles can leave a subnet with supposedly restrictive boundaries, and substantial administrative effort is required to avoid this. Organizations wishing to control such leakage are strongly advised to designate a small number of official gateways to handle all news exchange with the outside world (however, making such gateways too restrictive can also encourage the setting up of unofficial paths which can be exceedingly hard to track down). The sendme control message (5.4), insofar as it is still used, can be used to request articles with a given message identifier, even one that is not supposed to be supplied to the requestor. 7.2. Attacks 7.2.1. Denial of Service The proper functioning of individual newsgroups can be disrupted by the massive posting of "noise" articles, by the repeated posting of identical or near identical articles, by posting followups unrelated to their precursors, or which quote their precursors in full with the addition of minimal extra material (especially if this process is iterated), and by crossposting to, or setting followups to, totally unrelated newsgroups. Many have argued that "spam", massively multiposted (and to a lesser extent massively crossposted) articles, usually for advertising purposes, also constitutes a DoS attack in its own regard. This may be so. Such articles intended to deny service, or other articles of an inflammatory nature, may also have their From or Reply-To addresses set to valid but incorrect email addresses, thus causing large volumes of email to descend on the true owners of those addresses. Similar effects could be caused by any email header field which could cause every reading agent receiving it to take some externally visible action. For example, the Disposition-Notification-To header field defined in [RFC 2298] could cause huge numbers of acknowledgements to be emailed to an unsuspecting third party (for which reason [RFC 2298] declares that that header field SHOULD NOT be used in Netnews). It is a violation of this standard for a poster to use as his address a which he is not entitled to use. Even addresses with an invalid but a valid can cause disruption to the administrators of such domains. Posters who wish to remain anonymous or to prevent automated harvesting of their addresses, but who do not C. H. Lindsey [Page 44] News Article Architecture and Protocols November 2006 care to take the additional precautions of using more sophisticated anonymity measures, should avoid that violation by the use of addresses ending in the ".invalid" top-level-domain (see 6.5). A malicious poster may also prevent his article being seen at a particular site by preloading that site into the Path header field (F-3.1.5) and may thus prevent the true owner of a forged From or Reply-To address from ever seeing it. A malicious complainer may submit a modified copy of an article (e.g. with an altered Injection-Info header field) to the administrator of an injecting agent in an attempt to discredit the author of that article and even to have his posting privileges removed. Administrators should therefore obtain a genuine copy of the article from their own storage agent before taking such precipitate action. Administrative agencies with responsibility for establishing policies in particular hierarchies can and should set bounds upon the behaviour that is considered acceptable within those hierarchies (for example by promulgating charters for individual newsgroups, and other codes of conduct). Whilst this standard places an onus upon injecting agents to bear responsibility for the misdemeanours of their posters (which includes non-adherence to established policies of the relevant hierarchies as provided in section 6.2), and to provide assistance to the rest of the network by making proper use of the Injection-Info (F-3.2.8) header field, it makes no provision for enforcement, which may in consequence be patchy. Nevertheless, injecting sites which persistently fail to honour their responsibilities or to comply with generally accepted standards of behaviour are likely to find themselves blacklisted, with their articles refused propagation and even subject to cancellation, and other relaying sites would be well advised to withdraw peering arrangements from them. 7.2.2. Compromise of System Integrity The posting of unauthorized (as determined by the policies of the relevant hierarchy) control messages can cause unwanted newsgroups to be created, or wanted ones removed, from storage agents. Administrators of such agents SHOULD therefore take steps to verify the authenticity of such control messages, either by manual inspection (particularly of the Approved header field) or by checking any digital signatures that may be provided (see 5.1). In addition, they SHOULD periodically compare the newsgroups carried against any regularly issued checkgroups messages, or against lists maintained by trusted servers and accessed by out-of-band protocols such as FTP or HTTP. Malicious cancel messages (5.3) can cause valid articles to be removed from storage agents. Administrators of such agents SHOULD therefore take steps to verify that they originated from the (apparent) poster, the injector or the moderator of the article, or that in other cases they came from a place that is trusted to work C. H. Lindsey [Page 45] News Article Architecture and Protocols November 2006 within established policies and customs. Such steps SHOULD include the checking of any digital signatures, or other security devices, that may be provided (see 5.1). Articles containing Supersedes header fields (F-3.2.12) are effectively cancel messages, and SHOULD be subject to the same checks. Currently, many sites choose to ignore all cancel messages on account of the difficulty of conducting such checks. Improperly configured injecting and relaying agents can allow articles posted to moderated groups onto the net without first being approved by the moderator. Injecting agents SHOULD verify that moderated articles were received from one of the entities given in their Approved header fields and/or check any digital signatures that may be provided (see 5.1). There may be weaknesses in particular implementations that are subject to malicious exploitation. In particular, it has not been unknown for complete shell scripts to be included within Control header fields. Implementors need to be aware of this. Reading agents should be chary of acting automatically upon MIME objects with an "application" Content-Type that could change the state of that agent, except in contexts where such applications are specifically expected (as in 4). Even the Content-Type "text/html" could have unexpected side effects on account of embedded objects, especially embedded executable code or URIs that invoke non-news protocols such as HTTP [RFC 2616]. It is therefore generally recommended that reading agents do not enable the execution of such code (since it is extremely unlikely to have a valid application within Netnews) and that they only honour URIs referring to other parts of the same article. Non-printable characters embedded in article bodies may have surprising effects on printers or terminals, notably by reconfiguring them in undesirable ways which may become apparent only after the reading agent has terminated. 7.3. Liability [This whole section might be better removed to [USEAGE].] There is a presumption that a poster who sends an article to Usenet intends it to be stored on a multitude of storage agents, and has therefore given permission for it to be copied to that extent. Nevertheless, Usenet is not exempt from the Copyright laws, and it should not be assumed that permission has been given for the article to be copied outside of Usenet, nor for its permanent archiving contrary to any Archive header field that may be present. Posters also need to be aware that they are responsible if they breach Copyright, Libel, Harassment or other restrictions relating to material that they post, and that they may possibly find themselves liable for such breaches in jurisdictions far from their own. Serving agents may also be liable in some jurisdictions, especially if the C. H. Lindsey [Page 46] News Article Architecture and Protocols November 2006 breach has been explicitly drawn to their attention. Users who are concerned about such matters should seek advice from competent legal authorities. 8. IANA Considerations IANA is requested to register the following media types, described elsewhere in this standard, for use with the Content-Type header field, in the IETF tree in accordance with the procedures set out in [RFC 4288]. application/news-transmission (4.1) application/news-groupinfo (4.3) application/news-checkgroups (4.4) IANA is also requested to change the status of the following media type to "OBSOLETE". message/news (4.2) NOTE: "Application/news-transmission" is an update, with clarification and additional optional parameters, to an existing registration. "Message/rfc822" should now be used in place of the obsoleted "message/news". nr H1 7 9. References 9.1. Normative References [ANSI X3.4] "American National Standard for Information Systems - Coded Character Sets - 7-Bit American National Standard Code for Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986. [RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names", RFC 2606, June 1999. [RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April 2001. [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration procedures for message header fields", RFC 3864. [USEFOR] K. Murchison et al, "News Article Format", draft-ietf- usefor-usefor-11.txt. [USEPRO] This Standard. C. H. Lindsey [Page 47] News Article Architecture and Protocols November 2006 9.2. Informative References [PGPVERIFY] David Lawrence, . [RFC 1036] M. Horton and R. Adams, "Standard for Interchange of USENET Messages", RFC 1036, December 1987. [RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. [RFC 2046] N. Freed and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. [RFC 2142] D. Crocker, "Mailbox Names for Common Services, Roles and Functions", RFC 2142, May 1997. [RFC 2298] R. Fajman, "An Extensible Message Format for Message Disposition Notifications", RFC 2298, March 1998. [RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [RFC 2821] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer Protocol", RFC 2821, April 2001. [RFC 3977] C. Feather, "Network News Transport Protocol (NNTP)", RFC 3977. [RFC 4288] N. Freed and J. Klensin, "Media Type Specifications and Registration Procedures", RFC 4288, December 2005. [RFC 976] Mark R. Horton, "UUCP mail interchange format standard", RFC 976, February 1986. [Son-of-1036] Henry Spencer, "News article format and transmission", , June 1994. [USEAGE] draft-ietf-usefor-useage-*.txt. 10. Acknowledgements As this document is the result of a ten year effort, the number of people that have contributed to its content are too numerous to mention individually. Many thanks go out to all past and present members of the USEFOR Working Group of the Internet Engineering Task Force (IETF) and the accompanying mailing list. C. H. Lindsey [Page 48] News Article Architecture and Protocols November 2006 11. Contact Address Editor Charles. H. Lindsey 5 Clerewood Avenue Heald Green Cheadle Cheshire SK8 3JU United Kingdom Phone: +44 161 436 6131 Email: chl@clerew.man.ac.uk [ Working group chairs Alexey Melnikov Harald Tveit Alvestrand ] Comments on this draft should preferably be sent to the mailing list of the Usenet Format Working Group at ietf-usefor@imc.org. Appendix A - Obsolete Control Messages This present standard obsoletes certain control messages defined in [RFC 1036] (see 5.5), all of which had the effect of requesting a description of a relaying or storage agent's software, or its peering arrangements with neighbouring sites, to be emailed to the article's reply address. Whilst of some utility when Usenet was much smaller than it is now, they had become no more than a tool for the malicious sending of mailbombs. Moreover, many organizations now consider information about their internal connectivity to be confidential. version sendsys whogets senduuname "Version" requested details of the transport software in use at a site. "Sendsys" requested the full list of newsgroups taken, and the peering arrangements. "Whogets" was similar, but restricted to a named newsgroup. "Senduuname" resembled "sendsys" but restricted to the list of peers connected by UUCP. Historically, a checkgroups body consisting of one or two lines, the first of the form "-n newsgroup", caused check-groups to apply to only that single newsgroup. C. H. Lindsey [Page 49] News Article Architecture and Protocols November 2006 Historically, an article posted to a newsgroup whose name had exactly three components of which the third was "ctl" signified that article was to be taken as a control message. The Subject header field specified the actions, in the same way the Control header field does now. These forms are documented for archaeological purposes only; they MUST NO LONGER be used. Appendix B - Differences from the Protocols in RFC 1036 and its derivatives This apendix contains a list of changes that have been made to the protocols of the earlier standards, specifically [RFC 1036]. See F- Appendix B. for the related syntax changes. o There is a new Control message 'mvgroup' to facilitate moving a group to a different place (name) in a hierarchy. o Certain Control messages (Appendix A) have been made obsolete, and the special significance of "cmsg" when at the start of a Subject header field has been removed (section 5). o Additional media types are defined for better structuring of control messages (4.3 and 4.4). o Distributions are expected to be checked at the receiving end, as well as the sending end, of a relaying link. o There are numerous other small changes, clarifications and enhancements. Appendix C - Transitional Arrangements It is the intention that the new features of [USEFOR] and of this standard can be assimilated into Usenet as it presently operates without major interruption to the service, though some of the new features may not begin to show benefit until they become widely implemented. An important distinction must be made between news servers, which are responsible for the distribution and storage of news articles, and user agents, which are responsible for interactions with users. It is important that the former should be upgraded to conform to this standard as soon as possible to provide the benefit of the enhanced facilities. Fortunately, the number of distinct implementations of such servers is rather small, at least so far as the main "backbone" of Usenet is concerned, and many of the new features are already supported. Contrariwise, there are a great number of implementations of user agents, installed on a vastly greater number of small sites. Therefore, the new functionality has been designed so that existing user agents may continue to be used, although the full benefits may not be realised until a substantial proportion of them have been upgraded. In the list which follows, care has been taken to distinguish the implications for both kinds of agent. C. H. Lindsey [Page 50] News Article Architecture and Protocols November 2006 o The introduction of whitespace and folding into the Newsgroups and related header fields (F-3.1.4, F-3.2.4, F-3.2.6) and of s into the References header field (F-3.2.10) will cause problems for existing agents, and [USEFOR] provides that they SHOULD NOT be generated at the present time. o The requirement in [USEFOR] to support MIME reflects a practice that is already widespread. Articles in strict compliance with the previous standards (using strict US-ASCII) will be unaffected. o All the header fields newly introduced by [USEFOR] can safely be ignored by existing software, albeit with loss of the new functionality. o The new style of Path header field, using a to allow "!!" as a and introducing s (which can be distinguished from s by their leading ".") is already consistent with the previous standards. However, the intention is that relaying agents should eventually reject articles in the old style, and so this possibility should be offered as a configurable option in relaying agents. User agents are unaffected. o The new Control: mvgroup command will need to be implemented in storage agents. For the benefit of older storage agents it is therefore RECOMMENDED that it be followed shortly by a corresponding newgroup command and it MUST always be followed by a rmgroup command for the old group after a reasonable overlap period. An implementation of the mvgroup command as an alias for the newgroup command would thus be minimally conforming. User agents are unaffected. o Provision is made for relaying and storage agents to use the Date header field in the case of articles injected through existing agents which do not yet provide an Injection-Date header field. Appendix D - Notices Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. C. H. Lindsey [Page 51] News Article Architecture and Protocols November 2006 The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf- ipr@ietf.org. Full Copyright Statement Copyright (C) The IETF Trust (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." Appendix E - Change Log [This Appendix is to be removed prior to final publication.] For version 01 1 Numerous texts describing protocol features related to particular header fields in parts of [ARTICLE] which were destined to become part of [USEFOR] have been moved to appropriate locations within section 7 of this document. Such revised texts will be found in sections 7.2.2 Steps 4, 6, 7, 10, 11, 12; 7.2.3 Step 1(b); 7.3 introductory paragraphs, Steps 1, 4, 8, 9, and some final paragraphs; 7.4 introductory and final paragraphs; 7.9.1 Step 5. 2 A section on "Duties of a Reading Agent" (7.8) has been added. 3 Some demotions MUST -> SHOULD -> MAY, as noted in pseudo- comments, have been made or proposed in sections 7.3 7.3 Step 4. 4 Part of the procedure for examining Path header fields by relaying agents has been moved to storage agents, as explained in pseudo-comments in section 7.4. 5 Some renumbering of sections and minor textual clarifications. For version 02 C. H. Lindsey [Page 52] News Article Architecture and Protocols November 2006 1 2nd para. of a-7 temporarily reinstated in section 6. 2 Para. in section 6 relating to propagation of control messages and local policy removed to [USEAGE].] 3 Requirement for some relaying agents to examine control messages for non-existent groups 6 7.3 4 Text regarding "aliasing out" brought into line with actual practice. 7.3 5 More realistic wording regarding the expectations of reading agents 7.7 7.4 6 "Precursor" is now defined for all cases in which a References header field may be used (even though "followup" is not always defined under Alternative-1). 2.1 7 Provision is made for a poster to use a mailbox ending in ".invalid" in a From header field (formerly in a-5.2). 7.5 8 "Inheritable" and "Variant" header fields defined (formerly in a-4.2.5). 2.3 9 Additional wording regarding function of verb/arguments/body in control messages (formerly in a-6.13). 6 10 NOTE regarding not altering message indentifiers during transport or copying added (formerly in a-5.3). 7.3 11 All mention of MIME-style parameters and extension-parameters removed. 3.1 7.6 For version 03 1 The term "inheritable header" is no longer defined. Instead, the term "inherited' is used in place of "taken" when defining the actions of a followup agent. 7.6 C. H. Lindsey [Page 53] News Article Architecture and Protocols November 2006 2 Consequent changes to "variant header field", and also mention of Injection-Info as sometimes variant. 2.3 3 The term "reply address" is no longer defined. 4 References now made to sections within USEFOR using "F-..." notation. 5 Cross-references to sections within USEFOR added. Consistent use of <...> around all mentions of syntactic objects. All occurrences of "Foobar-header" changed to "Foobar header". Many other minor textual changes. 6 changed to , to avoid confusion with "control message", which signifies the complete article containing the . 7 has been changed to (since the earlier practice of multiple arguments is now deprecated). Likewise . For version 04 1 References divided into Normative and Informational. 2 All mention of the Mail-Copies-To, Posted-And-Mailed and Complaints-To header fields removed. 3 NOTE added to contrast MUST for References header field with SHOULD in RFC 2822. 7.6 4 Changes arising from the new syntax of s and s. 7.3 5 Changes to clarify the construction of the References header field. 7.6.1 6 Changes due to removal of s from further header fields. 7 New section on Identification of news-servers describing acceptable forms for s. 2.3 8 Definition of "semantic content" of a header field. 2.1 9 Systematic replacement of "header" by "header field". C. H. Lindsey [Page 54] News Article Architecture and Protocols November 2006 10 More stringent rules for checking s in control messages for compliance with USEFOR. 6.2 For version 05 1 Historical Appendices A.1 and A.2 removed, in anticipation of republication of Son-of-1036 as an Informational RFC. 1.3 2 Definitions of technical terms adopted from USEFOR rather than defining them separately. 3 Discussion of rewritten to reflect recent developments (but still awaiting further work in USEFOR). 2.3 4 Items now included in Appendix A of USEFOR have been removed from section 3, and the "Transitional Arrangements" (which still cover the USEFOR changes) have been modified to reflect this. For version 06 1 Procedures for constructing the Path header field updated to conform with USEFOR. 2 s/serving agent/storage agent/ throughout. 3 s/trusted source/verified source/ 4 Chapter 3 (Changes to the existing protocols) moved to Appendices B & C, and subsequent chapters renumbered. C. H. Lindsey [Page 55]