Network Working Group D. Kohn Internet-Draft Skymoon Ventures Updates: 2822 (if approved) March 19, 2003 Obsoletes: 1036 (if approved) Expires: September 17, 2003 News Article Format draft-kohn-news-article-02.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on September 17, 2003. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This document defines the format of network news articles. This is the first of four documents that update and obsolete RFC 1036. This document focuses on the syntax semantics of network news articles. RFC XXXX is also a standards-track document describing the protocol issues for network news transmission, independent of protocols such as NNTP and IMAP. An informational document, RFC YYYY, describes recommendations to improve interoperability and usability. The fourth document, RFC ZZZZ, an experimental document, specifies internationalization of message headers. Network news articles resemble mail messages but are broadcast to Kohn Expires September 17, 2003 [Page 1] Internet-Draft News Article Format March 2003 potentially large audiences, using a flooding algorithm that propagates one copy to each interested host (or group thereof), typically stores only one copy per host, and does not require any central administration or systematic registration of interested users. Network news originated as the medium of communication for Usenet, circa 1980. Since then Usenet has grown explosively, and many Internet sites participate in it. In addition, the news technology is now in widespread use for other purposes, on the Internet and elsewhere. This document defines the format of network news articles in the context of the Internet Message Format, and adds MIME support for multimedia and internationalized message bodies. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . 4 1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 Requirements Notation . . . . . . . . . . . . . . . . . . 4 1.3 Errata . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Syntax Notation . . . . . . . . . . . . . . . . . . . . . 5 1.5 Definitions . . . . . . . . . . . . . . . . . . . . . . . 5 1.6 Structure of This Document . . . . . . . . . . . . . . . . 5 2. Format . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1 Base . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 MIME Conformance . . . . . . . . . . . . . . . . . . . . . 6 2.3 Other MIME Support . . . . . . . . . . . . . . . . . . . . 6 3. Headers . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1 New Internet Message Format Headers . . . . . . . . . . . 7 3.2 Mandatory Headers . . . . . . . . . . . . . . . . . . . . 7 3.3 Date . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.4 Subject . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.5 Message-ID . . . . . . . . . . . . . . . . . . . . . . . . 8 3.6 News Headers . . . . . . . . . . . . . . . . . . . . . . . 9 3.6.1 Newsgroups . . . . . . . . . . . . . . . . . . . . . . . . 9 3.6.2 Path . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.6.3 Followup-To . . . . . . . . . . . . . . . . . . . . . . . 10 3.6.4 Expires . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.6.5 Control . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.6.6 Distribution . . . . . . . . . . . . . . . . . . . . . . . 11 3.6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.6.8 Approved . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.6.9 Organization . . . . . . . . . . . . . . . . . . . . . . . 12 3.6.10 Xref . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.6.11 Supersedes . . . . . . . . . . . . . . . . . . . . . . . . 12 3.7 Other Message Headers . . . . . . . . . . . . . . . . . . 12 3.7.1 References . . . . . . . . . . . . . . . . . . . . . . . . 12 4. Internationalization Considerations . . . . . . . . . . . 13 Kohn Expires September 17, 2003 [Page 2] Internet-Draft News Article Format March 2003 5. Security Considerations . . . . . . . . . . . . . . . . . 14 Normative References . . . . . . . . . . . . . . . . . . . 15 Informative References . . . . . . . . . . . . . . . . . . 16 Author's Address . . . . . . . . . . . . . . . . . . . . . 17 A. Acknowledgements . . . . . . . . . . . . . . . . . . . . . 18 Intellectual Property and Copyright Statements . . . . . . 19 Kohn Expires September 17, 2003 [Page 3] Internet-Draft News Article Format March 2003 1. Introduction 1.1 Scope "Netnews" is a set of protocols for generating, storing and retrieving news "articles" (which use the Internet Message Format) and for exchanging them among a readership which is potentially widely distributed. It is organized around "newsgroups", with the expectation that each reader will be able to see all articles posted to each newsgroup in which she participates. These protocols most commonly use a flooding algorithm which propagates copies throughout a network of participating servers. Typically, only one copy is stored per server, and each server makes it available on demand to readers able to access that server. The predecessor to this document [RFC1036] said that: "In any situation where this standard conflicts with the Internet [email standard, the latter] should be considered correct and this standard in error." The basic philosophy of this document follows that previous convention, so as to standardize news article syntax firmly in the context of Internet Message Format syntax. In the context of the Internet messaging architecture, different protocols (such as IMAP [RFC2060], POP3 [RFC1939], NNTP [RFC0977] and SMTP [RFC2821]) are seen as alternative ways of moving around the same content. That content is the Internet Message Format as specified by [RFC2822], including optional enhancements such as MIME [RFC2049]. A user should be able to ingest an article via NNTP, read it via IMAP, forward it off to someone else via SMTP and have them read it via POP3 all without having to alter the content. This document uses a cite by reference methodology, rather than trying to repeat the contents of other standards, which could otherwise result in subtle differences and interoperability challenges. Although this document is as a result rather short, it requires complete understanding and implementation of the normative references to be compliant. 1.2 Requirements Notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 1.3 Errata The RFC Editor makes available errata for RFCs at [errata]. Implementers should review that page for normative references, noting in particular that errata currently exist for [RFC2046]. Kohn Expires September 17, 2003 [Page 4] Internet-Draft News Article Format March 2003 1.4 Syntax Notation Headers defined in this specification use the Augmented Backus-Naur Form (ABNF) notation (including the Core Rules) specified in [RFC2234] and many constructs (specifically , , , and ) defined in [RFC2822]. Section 3.5 updates the [RFC2822] definition of . 1.5 Definitions Add definitions here for at least article, user agent, injector, moderator, server, and gateway. Agent refers generically to all roles. 1.6 Structure of This Document Section 2 defines the format of news articles. Section 3 defines some additional headers necessary for the netnews environment. Kohn Expires September 17, 2003 [Page 5] Internet-Draft News Article Format March 2003 2. Format 2.1 Base News articles MUST conform to the "legal to generate syntax" specified in Section 3 of [RFC2822]. News agents SHOULD also support the obsolete syntax specified in Section 4 of [RFC2822], particularly to support old news messages and gatewayed obsolete mail messages, but they MUST NOT generate such syntax. 2.2 MIME Conformance User agents MUST meet the definition of MIME-conformance in [RFC2049]. This level of MIME Conformance provides support for internationalization and multimedia in message bodies, and for receipt (but not generation) of internationalized headers. Generation of internationalized message headers is specified by RFC ZZZZ. 2.3 Other MIME Support User agents conformant with this document SHOULD support receipt (and automatic reassembly) of message/partial MIME messages, as specified in Section 5.2.2 of [RFC2046] and MAY support generation of message/ partial articles for excessively large articles. User agents SHOULD send regular paragraph text as "text/plain; format=flowed" as specified in [RFC2646] and SHOULD preserve flowed text (including quoting) when replying or forwarding, as described in that specification. User agents SHOULD support on receipt and MAY generate extension header fields, including but not limited to Content-Disposition [RFC2183] and Content-Language [RFC3282]. Kohn Expires September 17, 2003 [Page 6] Internet-Draft News Article Format March 2003 3. Headers 3.1 New Internet Message Format Headers The following header fields extend the fields defined in section 3.6 of [RFC2822] as follows: fields =/ *( newsgroups / path / followup-to / expires / control / distribution / summary / approved / organization / xref / supersedes ) Each of these headers may occur at most once in a news article. 3.2 Mandatory Headers Each news article conformant with this specification MUST have exactly one of each of the following headers: From, Subject, Message-ID, Date, Newsgroups, and Path. From is exactly as specified in [RFC2822]. Date and Subject are fully conformant with [RFC2822], though with extra implementation advice in Section 3.3 and Section 3.4, respectively. In Section 3.5, this document updates the construct from [RFC2822] so as to ensure that Internet Message Format Message-IDs are usable in widely deployed news software. Following [RFC2822] syntax, the headers defined in this document do not require a space between the ":" and the field's contents. (E.g., "Subject:Hello World" is acceptable, as opposed to requiring "Subject: Hello World".) To be compliant with this specification, news agents MUST support 0 or more spaces between the colon and the field's contents. However, to maximize compatibility with the installed base of news agents, implementers SHOULD use exactly one space. 3.3 Date The Date header is the same as that specified in Sections 3.3 and 3.6.1 of [RFC2822]. However, the use of "GMT" as a time zone, which is part of , is widespread in news articles today. Kohn Expires September 17, 2003 [Page 7] Internet-Draft News Article Format March 2003 Therefore, agents MUST accept, but MUST NOT generate, constructs where ="GMT". (As stated in Section 2.1, support for would otherwise have been SHOULD accept, MUST NOT generate.) Note that these requirements apply wherever is used, including Expires in Section 3.6.4. 3.4 Subject The Subject header contains a short string identifying the topic of the message. Section 3.6.5 of [RFC2822] says: When used in a reply, the field body MAY start with the string "Re: " (from the Latin "res", in the matter of) followed by the contents of the "Subject:" field body of the original message. If this is done, only one instance of the literal string "Re: " ought to be used since use of other strings or more than one instance can lead to undesirable consequences. Because of the importance of threading in news, that MAY is amplified to a SHOULD: Follow-ups to an article SHOULD begin with the subject "Re: " followed by the original subject of the referenced article (with that original subject stripped of any starting "Re: "). User agents MAY remove strings that are known to be used erroneously as back-reference (such as "Re(2): ", "Re:", "RE: ", or "Sv: ") from the beginning of the Subject field body when composing the subject of a followup, and add a correct back-reference in front of the result. User agents replying to a message MUST NOT use any other string except "Re: " as a back reference. Specifically, a translation of "Re: " into a local language or usage MUST NOT be used. User agents MAY present to the user a translation of "Re: ", but this MUST only be an artifact of the user interface and MUST NOT be part of the actual news article. 3.5 Message-ID The "Message-ID:" field contains a single unique message identifier. This is the only header field definition that updates [RFC2822]. The ABNF should be used as below, but the requirements and descriptive text from Section 3.6.4 of [RFC2822] still apply. Kohn Expires September 17, 2003 [Page 8] Internet-Draft News Article Format March 2003 message-id = "Message-ID:" msg-id CRLF msg-id = [CFWS] msg-id-core [CFWS] msg-id-core = "<" id-left "@" id-right ">" ; maximum length is 250 octets id-left = dot-atom-text / no-fold-quote / obs-id-left id-right = dot-atom-text / no-fold-literal / obs-id-right no-fold-quote = DQUOTE *( qtext / no-space-qp ) DQUOTE no-fold-literal = "[" *( htext / no-space-qp ) "]" no-space-qp = ( "\" ptext ) / obs-qp ptext = %d33-61 / ; Printable characters excluding ">" %d63-126 / obs-text htext = HEXDIG / ; hexadecimal digits, case-insensitive "." / ; IPv4 separator ":" ; IPv6 separator Although compliant agents MUST support [CFWS] between the "Message-ID:" and the , implementers SHOULD generate exactly one space there, to maximize compatibility with the installed base. Note that this updated ABNF applies wherever is used, including the In-Reply-To and References headers mentioned in Section 3.7 and Section 3.7.1. 3.6 News Headers 3.6.1 Newsgroups The Newsgroups header specifies to which newsgroup(s) the article is posted. Kohn Expires September 17, 2003 [Page 9] Internet-Draft News Article Format March 2003 newsgroups = "Newsgroups:" newsgroup-list CRLF newsgroup-list = [FWS] newsgroup-name *( "," [FWS] newsgroup-name ) [FWS] newsgroup-name = component *( "." component ) ; 71 character max component = plain-component plain-component = component-start *29component-rest component-start = ALPHA / DIGIT component-rest = ALPHA / DIGIT / "+" / "-" / "_" A newsgroup name consists of one or more components separated by periods, with no more than 71 characters total. Each component consists of less than 30 or less letters and digits. 3.6.2 Path The Path header's content indicates which relayers the article has already visited, so that unnecessary redundant transmission can be avoided. path = "Path:" [FWS] *( path-host [FWS] path-delimiter [FWS] ) path-host [FWS] CRLF path-host = ( ALPHA / DIGIT ) *( ALPHA / DIGIT / "-" / "." / ":" / "_" ) path-delimiter = "!" 3.6.3 Followup-To The Followup-To header specifies to which newsgroup(s) followups should be posted. followup-to = "Followup-To:" ( newsgroup-list / poster-text ) CRLF poster-text = [FWS] %d112.111.115.116.101.114 [FWS] ; "poster" in lower-case The syntax is the same as that of the Newsgroups content, with the exception that the magic word "poster" (which is always lowercase) means that followups should be mailed to the article's reply address Kohn Expires September 17, 2003 [Page 10] Internet-Draft News Article Format March 2003 rather than posted. 3.6.4 Expires The Expires header content specifies a date and time when the article is deemed to be no longer useful and could usefully be removed ("expired"). expires = "Expires:" date-time CRLF 3.6.5 Control The Control header marks the article as a control message, and specifies the desired actions (additional to the usual ones of storing and/or relaying the article). The verb indicates what action should be taken, and the argument(s) (if any) supply details. In some cases, the body of the article may also contain details. Control messages are further specified in the companion document, RFC XXXX. control = "Control:" verb *( FWS argument ) CRLF An article with a Control header MUST NOT have a Supersedes header. 3.6.6 Distribution The Distribution header content specifies geographic or organizational limits on an article's propagation. distribution = "Distribution:" dist-name *( "," dist-name ) CRLF dist-name = [FWS] ALPHA / DIGIT *( ALPHA / DIGIT / "+" / "-" / "_" ) [FWS] "All" MUST NOT be used as a distribution-name. Distribution-names SHOULD contain at least three characters, except when they are two-letter country names as in [ISO.3166.1988]. Distribution-names are case-insensitive (i.e. "US", "Us" and "us" all specify the same distribution). 3.6.7 Summary The Summary header content is a short phrase summarizing the article's content. summary = "Summary:" unstructured CRLF Kohn Expires September 17, 2003 [Page 11] Internet-Draft News Article Format March 2003 3.6.8 Approved The Approved header content indicates the mailing addresses (and possibly the full names) of the persons or entities approving the article for posting. approved = "Approved:" mailbox-list CRLF 3.6.9 Organization The Organization header content is a short phrase identifying the poster's organization. organization = "Organization:" unstructured CRLF There is no "s" in Organization. 3.6.10 Xref The Xref header content indicates where an article was filed by the last relayer to process it. xref = "Xref:" [CFWS] path-host 1*( CFWS location ) [CFWS] location = newsgroup-name ":" 1*16DIGIT 3.6.11 Supersedes The Supersedes header content specifies articles to be cancelled. supersedes = "Supersedes:" 1*( [FWS] msg-id-core ) CRLF There is no "c" in Supersedes. 3.7 Other Message Headers The headers Reply-To, Sender, Comments, and Keywords are often used in news articles and have the identical meaning as that specified in [RFC2822]. In-Reply-To is also regularly used in news articles and has same the same meaning as that specified in [RFC2822], except that it uses the updated construct defined in Section 3.5. 3.7.1 References The References header has the same syntax as [RFC2822], except that it uses the updated construct defined in Section 3.5 and also adds the following requirements to the semantics. Kohn Expires September 17, 2003 [Page 12] Internet-Draft News Article Format March 2003 4. Internationalization Considerations Internationalization of news article bodies is provided using MIME mechanisms in Section 2.2. Generation of internationalized message headers is not specified in this document, and is instead specified in the experimental document, RFC ZZZZ. Kohn Expires September 17, 2003 [Page 13] Internet-Draft News Article Format March 2003 5. Security Considerations The news article format specified in this document does not provide any security services, such as confidentiality, authentication of sender, or non-forgery. Instead, such services need to be layered above, using such protocols as S/MIME [RFC2633] or PGP/MIME [RFC3156], or below, using secure versions of news transport protocols. Additionally, several currently non-standardized protocols [PGPVERIFY] will hopefully be standardized in the near future. Message-IDs (see Section 3.5) in news are required to be unique; articles are refused (in server-to-server transfer) if the ID has already been seen. So if you can predict the ID of a message, you can pre-empt it by posting a message (possibly to a quite different group) with the same ID, stopping your target message from propagating. Agents that generate message-ids for news articles SHOULD ensure that they are unpredictable. Kohn Expires September 17, 2003 [Page 14] Internet-Draft News Article Format March 2003 Normative References [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. [RFC2049] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Five: Conformance Criteria and Examples", RFC 2049, November 1996. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997. [RFC2646] Gellens, R., "The Text/Plain Format Parameter", RFC 2646, August 1999. [RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April 2001. Kohn Expires September 17, 2003 [Page 15] Internet-Draft News Article Format March 2003 Informative References [ISO.3166.1988] International Organization for Standardization, "Codes for the representation of names of countries, 3rd edition", ISO Standard 3166, August 1988. [PGPVERIFY] Lawrence, D., "PGPverify ", June 1999. [RFC0977] Kantor, B. and P. Lapsley, "Network News Transfer Protocol", RFC 977, February 1986. [RFC1036] Horton, M. and R. Adams, "Standard for interchange of USENET messages", RFC 1036, December 1987. [RFC1939] Myers, J. and M. Rose, "Post Office Protocol - Version 3", STD 53, RFC 1939, May 1996. [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text", RFC 2047, November 1996. [RFC2060] Crispin, M., "Internet Message Access Protocol - Version 4rev1", RFC 2060, December 1996. [RFC2183] Troost, R., Dorner, S. and K. Moore, "Communicating Presentation Information in Internet Messages: The Content-Disposition Header Field", RFC 2183, August 1997. [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and Languages", BCP 18, RFC 2277, January 1998. [RFC2633] Ramsdell, B., "S/MIME Version 3 Message Specification", RFC 2633, June 1999. [RFC2821] Klensin, J., "Simple Mail Transfer Protocol", RFC 2821, April 2001. [RFC3156] Elkins, M., Del Torto, D., Levien, R. and T. Roessler, "MIME Security with OpenPGP", RFC 3156, August 2001. [RFC3282] Alvestrand, H., "Content Language Headers", RFC 3282, May 2002. [errata] "RFC Editor Errata ". Kohn Expires September 17, 2003 [Page 16] Internet-Draft News Article Format March 2003 Author's Address Dan Kohn Skymoon Ventures 3045 Park Boulevard Palo Alto, California 94306 USA Phone: +1-650-327-2600 EMail: dan@dankohn.com URI: http://www.dankohn.com/ Kohn Expires September 17, 2003 [Page 17] Internet-Draft News Article Format March 2003 Appendix A. Acknowledgements Useful comments and/or text were provided by Mark Crispin, Claus Faerber, Ned Freed, Andrew Gierth, Tony Hansen, Paul Hoffman, Simon Josefsson, Bruce Lilly, Charles Lindsey, Ken Murchinson, Pete Resnick, and Henry Spencer. Kohn Expires September 17, 2003 [Page 18] Internet-Draft News Article Format March 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Kohn Expires September 17, 2003 [Page 19] Internet-Draft News Article Format March 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Kohn Expires September 17, 2003 [Page 20]