From bmwg-bounces@ietf.org Thu Jun 5 12:33:15 2008 Return-Path: X-Original-To: bmwg-archive@megatron.ietf.org Delivered-To: ietfarch-bmwg-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BB29028C1A6; Thu, 5 Jun 2008 12:33:15 -0700 (PDT) X-Original-To: bmwg@core3.amsl.com Delivered-To: bmwg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 958C03A6A49; Thu, 5 Jun 2008 12:23:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.767 X-Spam-Level: X-Spam-Status: No, score=-104.767 tagged_above=-999 required=5 tests=[AWL=-1.029, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_21=0.6, MIME_HTML_ONLY=1.457, MSGID_FROM_MTA_HEADER=0.803, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F05Ws5PrazQl; Thu, 5 Jun 2008 12:22:59 -0700 (PDT) Received: from mail121.messagelabs.com (mail121.messagelabs.com [216.82.245.115]) by core3.amsl.com (Postfix) with ESMTP id 225633A69CD; Thu, 5 Jun 2008 12:22:55 -0700 (PDT) X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-14.tower-121.messagelabs.com!1212693779!16360607!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.141] Received: (qmail 9886 invoked from network); 5 Jun 2008 19:23:00 -0000 Received: from sbcsmtp9.sbc.com (HELO flph161.enaf.ffdc.sbc.com) (144.160.128.141) by server-14.tower-121.messagelabs.com with AES256-SHA encrypted SMTP; 5 Jun 2008 19:23:00 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph161.enaf.ffdc.sbc.com (8.14.2/8.14.2) with ESMTP id m55JMv1F001660; Thu, 5 Jun 2008 12:22:58 -0700 Received: from klph001.kcdc.att.com (klph001.kcdc.att.com [135.188.3.11]) by flph161.enaf.ffdc.sbc.com (8.14.2/8.14.2) with ESMTP id m55JMqUH001518; Thu, 5 Jun 2008 12:22:52 -0700 Received: from kcdc.att.com (localhost.localdomain [127.0.0.1]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m55JMotv000941; Thu, 5 Jun 2008 14:22:52 -0500 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m55JMkvR000326; Thu, 5 Jun 2008 14:22:46 -0500 Message-Id: <200806051922.m55JMkvR000326@klph001.kcdc.att.com> Received: from acmt.att.com (unknown[135.210.112.118](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20080605192245gw100l7ocme>; Thu, 5 Jun 2008 19:22:46 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 05 Jun 2008 15:21:14 -0400 To: sunwq@sjtu.edu.cn, ccamp@ietf.org From: Al Morton In-Reply-To: <000001c8c168$2b657c90$823075b0$@edu.cn> References: <000001c8c168$2b657c90$823075b0$@edu.cn> Mime-Version: 1.0 X-Mailman-Approved-At: Thu, 05 Jun 2008 12:33:14 -0700 Cc: 'Rajiv Papneja' , ippm@ietf.org, 'xqwei' , pmol@ietf.org, 'zhangguoying' , sunwq@sjtu.edu.cn, 'Bin Gu' , bmwg@ietf.org, 'gjhhit' , 'blithe' Subject: Re: [bmwg] [ippm] Soliciting comments on GMPLS performance metrics draft in CCAMP X-BeenThere: bmwg@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Benchmarking Methodology Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0070497361==" Sender: bmwg-bounces@ietf.org Errors-To: bmwg-bounces@ietf.org --===============0070497361== Content-Type: text/html; charset="us-ascii" At 04:44 AM 5/29/2008, Weiqiang Sun wrote:

The authors would now like to solicit comments among the performance metric experts, as this document contains notations that are commonly seen in IPPM and BMWG. The authors believe that your expertise can help us further improve this document, with regard to the definition of the metrics and the structure as well. Please send your comments directly to the authors, or, to the CCAMP working group mail list (ccamp@ops.ietf.org). We do appreciate your input.

As I've said in the past, this draft is an excellent start,
and because it adopts the metric template used in bmwg and ippm,
it's an easy read. Thanks for doing that part.

more below,
Al

General comments:
-----------------
There is a taxonomy of possible outcomes for set-up protocols.
Each set-up attempt will have one of the following outcomes:

1. Success
2. Failure
3. Incorrect Set-up (wrong destination)

The same basic outcomes apply to release attempts.

In the current text, the set-up time metrics cover the
Successful outcome (they could be called Successful *.* Setup Time).

When a setup attempt fails, the setup time is Undefined. Stop there.
Please remove the "informally infinite" phrase between
parentheses, because it places the failure outcome on the
same dimension as success. As I've pointed out above,
the 3 outcomes are mutually exclusive. I know this text came
from IPPM RFCs, but this is a case where you should not follow
that precedent, IMHO. For more nit-picking background, read
http://tools.ietf.org/id/draft-morton-ippm-reporting-metrics-05.txt

Back to the Failure Outcome. I can imagine that there will be
technologies for which a metric on Setup Failure Ratio may be
more important than the Successful setup time - I may not care
if the setup takes 1 second or 10 seconds, but the 50% failure
ratio is annoying...

So, I propose to add metrics to cover the Failure outcome,
defined as a count of failures=Undefined setup times divided by the
total attempts. This would be a single outcome for a singleton
metric and a ratio for metrics with multiple LSP setups.
(I saw the metric in 12.4, it doesn't quite have the right wording
or emphasis - this metric should be a part of all setup and release
scenarios).

Adding metrics to cover the Incorrect setup/release outcome
would complete the taxonomy. I can accept that the Incorrect
outcome may be quite rare in practice, but having the metric
defined and ready is a reasonable use of resources. You can
note that these outcomes may be rare in some technologies as
part of the discussion.

Specific Comments:
------------------
(You can generalize these to all applicable sections)

3.3 Metric Parameters

o ID0, the ingress LSR ID

o ID1, the egress LSR ID

You should use these ID#s in the definition of the metric.

o T, a time

I've found that the two word description of a parameter is
not always sufficient for me. I suggest:

o T, a time when the setup is attempted

I suggest to add a parameter, for clarity:

o WAITTIME, the maximum waiting time for successful setup

and mention this parameter in the sections where discussed,
like 3.5 and 3.6.

----------

13. Discussion

The control-plane-only nature of these metrics should be clear(er)
in the Introduction/Scope.

Question:
How important is it to have data-plane verification of the
apparent control-plane success/failure/incorrect performance?

--===============0070497361== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg --===============0070497361==-- From bmwg-bounces@ietf.org Sun Jun 8 17:15:52 2008 Return-Path: X-Original-To: bmwg-archive@megatron.ietf.org Delivered-To: ietfarch-bmwg-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6BB5F3A68B6; Sun, 8 Jun 2008 17:15:52 -0700 (PDT) X-Original-To: bmwg@core3.amsl.com Delivered-To: bmwg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 975E23A68B6 for ; Sun, 8 Jun 2008 17:15:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.999 X-Spam-Level: X-Spam-Status: No, score=-0.999 tagged_above=-999 required=5 tests=[BAYES_50=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q8PCdp8xvbpI for ; Sun, 8 Jun 2008 17:15:50 -0700 (PDT) Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by core3.amsl.com (Postfix) with ESMTP id 11DD53A67CF for ; Sun, 8 Jun 2008 17:15:50 -0700 (PDT) Received: from compute1.internal (compute1.internal [10.202.2.41]) by out1.messagingengine.com (Postfix) with ESMTP id 03931106C0D; Sun, 8 Jun 2008 20:16:07 -0400 (EDT) Received: from heartbeat2.messagingengine.com ([10.202.2.161]) by compute1.internal (MEProxy); Sun, 08 Jun 2008 20:16:06 -0400 X-Sasl-enc: oAkCvDw4bMA2JxEqblethqj3ONDLx5Kmbi0zKHKT5fSU 1212970566 Received: from [192.168.1.127] (24-236-136-27.dhcp.bycy.mi.charter.com [24.236.136.27]) by mail.messagingengine.com (Postfix) with ESMTPSA id 56FFD2971A; Sun, 8 Jun 2008 20:16:06 -0400 (EDT) Message-Id: <9B4FD292-6805-414E-99AE-B47E9C4C04B6@wjcerveny.com> From: Bill Cerveny To: bmwg@ietf.org Mime-Version: 1.0 (Apple Message framework v919.2) Date: Sun, 8 Jun 2008 20:16:05 -0400 X-Mailer: Apple Mail (2.919.2) Cc: jkarthik@cisco.com, sporetsky@nextpointnetworks.com, Samir Vapiwala , Rajiv Papneja Subject: [bmwg] Comments on draft-ietf-bmwg-protection-term-04.txt X-BeenThere: bmwg@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Benchmarking Methodology Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Sender: bmwg-bounces@ietf.org Errors-To: bmwg-bounces@ietf.org My comments on draft-ietf-bmwg-protection-term-04.txt are below. I = hope you find them useful and please feel free to contact me if you = have any questions or need clarification. Bill Cerveny -------------------------------------- Overall, Bill=92s over-capitalization rant applies ;-) In general, there = is way too much capitalization. Terms such as "working path", = "protection," "backup path" or "path" should not be capitalized. In = particular, look at IEEE reference on acronyms: http://www.computer.org/por= tal/pages/ieeecs/publications/author/style/acronyms.html Abstract: =93The performance benchmarks are measured at the IP-Layer, so avoid = dependence on specific sub-IP protection mechanisms.=94 -- Consider = rewording =93=85 IP-Layer, so avoid=94 to something like =93=85 IP-layer, = avoiding=85=94 or =93=85 IP-layer and avoid =85=94 Introduction: (General comment =85 is this too much detail for an introduction??) =93Fast convergence times are critical to maintain reliable network = connectivity and performance.=94 =97 Consider changing to something like = =93Fast convergence times are critical for maintaining reliable network = connectivity and performance.=94 (Optional change) =93Technologies that function at sub-IP layers can be enabled to provide = further protection of IP traffic by providing the failure recovery at = the sub-IP layers so that the outage is not observed at the IP-layer.=94 = =97 Reword to something like: =93Technologies that provide failure = recovery at the sub-IP layers can further protect IP traffic such that = an outage is not observed at the IP layer.=94 (Not perfect, but better, = in my opinion) (Side observation: which =93layer=94 standard are you referring to? Is = this standard listed in references?) s/Benchmarking terminology have been defined/Benchmarking terminology = has been defined/ =93New terminology and methodologies specific to benchmarking sub-IP = layer protection mechanisms are required. This will enable different = implementations of the same protection mechanisms to be benchmarked = and evaluated.=94 =97 Perhaps change to =93New terminology and = methodologies specific to benchmarking sub-IP layer protection = mechanisms will enable different implementations of the same = protection mechanisms to be benchmarked and evaluated.=94 New paragraph?: =93The purpose of this document =85=94 New paragraph, changed text: =93The sequence of events is:=94 =93The Protection MAY=85=94?? Do you mean =93Protection MAY =85=94 or =93T= he = protection system MAY=85=94 Terminology Section Path: Why is L12 the first link and not something like L1? =93The path is defined in the sub-IP layer in this document, unlike an = IP path in RFC 2026 [1].=94 Perhaps change to =93A path as defined in this = document refers to a sub-IP layer path, unlike an IP path in RFC 2026 = [1].=94 Backup Path: s/The Backup Path SHALL be the Working Path/The backup path SHALL = become the working path/ Unclear =97 =93A new Path computed after the Failover Event is simply = Convergence [7] to a new Primary Path. =93 Disjoint Path: S/Discussions/Discussion/ :-) Point of Local Repair: s/Based on the functionality of the PLR, its role is defined based on = the type of method used./The role of the PLR is based on the type of = method used./ s/In the case the facility backup method is used/Where the facility = backup method is used/ Link, Node and Path Protection definitions: To me, at least as they = appear, =93protection=94 appears to be a =93condition=94 and not something = physical such as a path (although they may describe the impact of a = path). These definitions need to be defined as something like = =93condition where =85=94 or =93state where =85=94. You may be able to conv= ince me = otherwise, but these definitions look initially awkward to me. Redundant Node Protection: Are VRRP and HA defined or referenced? Expand on acronym on first = usage. Protected Interface: s/A Protected Interface is an interface protected by a Protection = Switching Systems/A protected interface is an interface protected by a = protection switching system/ (protection system system should be = singular) Non-Protection System Node: S/A node that is not capable of participating in a Protection = Switching System, however it MAY exist along the Primary Path or = Backup Path./A node that is not capable of participating in a = protection switching system. However, it MAY exist along the primary = path or backup path./ Headend Node: Discussion begins and ends by discussing =93headend=94 node, but in = between, discussion is regarding =93failover=94 nodes. What is = relationship between headend and failover nodes? (Side comment: Acronym PLR is used infrequently enough that I have to = think back to what the acronym refers to) Packet Loss Based Method: =93Offered load rate=94 not defined, as far as I can tell. Additionally, = the offered load rate measurement units aren=92t defined (which helps in = understanding equation #3). Finally, the line =93Units are =85=94 appears= = to state units are seconds, but =93measurement units=94 are stated as = milliseconds Time Based Method: Simplify, shorten or break-up: =93The purpose of this method is to quantify the duration of failure = or reversion on a time scale with granularity of milliseconds based on = the observation of unimpaired packets, using Equation 2 with the = difference being that the time values are obtained from the timestamp = in the packet payload rather than from the Tester.=94 _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg From bmwg-bounces@ietf.org Wed Jun 11 06:42:58 2008 Return-Path: X-Original-To: bmwg-archive@megatron.ietf.org Delivered-To: ietfarch-bmwg-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CCE6C3A68FD; Wed, 11 Jun 2008 06:42:58 -0700 (PDT) X-Original-To: bmwg@core3.amsl.com Delivered-To: bmwg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 25B003A6980 for ; Wed, 11 Jun 2008 06:42:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sBB+-eQlWOwV for ; Wed, 11 Jun 2008 06:42:56 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by core3.amsl.com (Postfix) with ESMTP id 4ED4D3A6838 for ; Wed, 11 Jun 2008 06:42:56 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.27,624,1204498800"; d="scan'208";a="11405181" Received: from ams-dkim-1.cisco.com ([144.254.224.138]) by ams-iport-1.cisco.com with ESMTP; 11 Jun 2008 15:43:20 +0200 Received: from ams-core-1.cisco.com (ams-core-1.cisco.com [144.254.224.150]) by ams-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id m5BDhKTN011895 for ; Wed, 11 Jun 2008 15:43:20 +0200 Received: from xbh-ams-332.emea.cisco.com (xbh-ams-332.cisco.com [144.254.231.87]) by ams-core-1.cisco.com (8.13.8/8.13.8) with ESMTP id m5BDhKwK021313 for ; Wed, 11 Jun 2008 13:43:20 GMT Received: from xmb-ams-337.cisco.com ([144.254.231.82]) by xbh-ams-332.emea.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 11 Jun 2008 15:43:20 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 11 Jun 2008 15:43:19 +0200 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [bmwg] [answer to Bill Cerveny] RE: New Version Notification for draft-janovak-bmwg-ipflow-meth-00 [answer to Bill Cerveny] Thread-Index: Aci7LkZh+gO+HZNLRsGCDeUXVuHjVwAADHzABCYkuIA= From: "Jan Novak (janovak)" To: X-OriginalArrivalTime: 11 Jun 2008 13:43:20.0526 (UTC) FILETIME=[1C644EE0:01C8CBC9] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=4604; t=1213191800; x=1214055800; c=relaxed/simple; s=amsdkim1002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=janovak@cisco.com; z=From:=20=22Jan=20Novak=20(janovak)=22=20 |Subject:=20[bmwg]=20[answer=20to=20Bill=20Cerveny]=20RE=3A =20New=20Version=20Notification=20for=20draft-janovak-bmwg-i pflow-meth-00=20[answer=20to=20Bill=20Cerveny] |Sender:=20; bh=ZvJ/8/Jzft+Mj0JD2Lv+936aFsl0q2cTTChtxCKWj5U=; b=JfNdp26LKF63sws2tHl78L/jIh012BCq/oIRG04zddWwipoggQr42L5F82 IvGKjrDXqkZnLpabTz3vcKmiAt6bCvG7iYfh7ek5WDa61fbXmKmb+vpvHD8R Tot1eXchMk; Authentication-Results: ams-dkim-1; header.From=janovak@cisco.com; dkim=pass ( sig from cisco.com/amsdkim1002 verified; ); Subject: [bmwg] [answer to Bill Cerveny] RE: New Version Notification for draft-janovak-bmwg-ipflow-meth-00 [answer to Bill Cerveny] X-BeenThere: bmwg@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Benchmarking Methodology Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: bmwg-bounces@ietf.org Errors-To: bmwg-bounces@ietf.org Hi Bill, I am sorry, was on holidays and missed your answer .... >devices, the throughput measurement wouldn't be appropriate. Perhaps >the scope of this document should be specifically on network devices >that transit network traffic? Yes, the intention was to make it specific for only network devices. Will review the text to make sure it is clear. >Wouldn't a big consideration be the quality and accuracy of the flow >data? How would the accuracy of the generated flow be evaluated? The >flow generating capabilities of a network device is certainly less >useful if the device's flow accuracy degrades when it is transiting >more traffic. I would consider this out of the scope since to check that the testing would have to involve some external collector and the way those handle data is even more complex than what happens on the network devices when just registering the traffic. I will think about it yet, but based on the testing we have done I don't quite see how to do it. > What about flow sampling? There is no reference to sampling in the > draft. Yes, that should not be difficult to discuss. Thank you for your comments. Jan The climate of Edinburgh is such that the weak succumb young .... and the strong envy them. Dr. Johnson Jan, I think a document on benchmarking flow analysis and export could be valuable. Questions: - The draft assumes the device generating flow will be a transit device (such as a router). What about flow generating devices that passively monitor the network and are not transit devices? For such devices, the throughput measurement wouldn't be appropriate. Perhaps the scope of this document should be specifically on network devices that transit network traffic? - Wouldn't a big consideration be the quality and accuracy of the flow data? How would the accuracy of the generated flow be evaluated? The flow generating capabilities of a network device is certainly less useful if the device's flow accuracy degrades when it is transiting more traffic. - What about flow sampling? There is no reference to sampling in the draft. Thanks, Bill Cerveny > -----Original Message----- > From: Jan Novak (janovak) > Sent: 21 May 2008 11:45 > To: 'bmwg@ietf.org' > Subject: FW: New Version Notification for > draft-janovak-bmwg-ipflow-meth-00 > > Hi, > > I have worked for over 3 years in the IP Flow (RFC5101) testing area > (RFC3954 - Netflow) and during that time we have got numerous requests > from various customers about Netflow performance. Every time > it started off a long discussion what was meant by the term > "netflow performance", how to measure it and deliver some > reproducible data. > We ended up compiling an internal document which we provided > to those asking the questions above. This IETF draft document > tries to specify IP Flow performance and some characteristics to > quantify and measure it in an implementation independent way. > > I would appreciate the opinion of this working group if this bit > of work is deemed useful and worthwhile of accepting as the WG > document. > > Thanks, Jan > > The climate of Edinburgh is such that the weak > succumb young .... and the strong envy them. > > Dr. Johnson > > > -----Original Message----- > > From: IETF I-D Submission Tool [mailto:idsubmission@ietf.org] > > Sent: 21 May 2008 11:34 > > To: Jan Novak (janovak) > > Subject: New Version Notification for > > draft-janovak-bmwg-ipflow-meth-00 > > > > > > A new version of I-D, draft-janovak-bmwg-ipflow-meth-00.txt > > has been successfuly submitted by Jan Novak and posted to the > > IETF repository. > > > > Filename: draft-janovak-bmwg-ipflow-meth > > Revision: 00 > > Title: IP Flow Information Accounting and > > Export Benchmarking Methodology > > Creation_date: 2008-05-21 > > WG ID: Independent Submission > > Number_of_pages: 19 > > > > Abstract: > > This document provides methodology and framework for quantifying > > performance implications of enabling selective monitoring of > > IP flows on a network device and export of this information to > > a collector as specified in [RFC5101]. > > > > > > > > > > The IETF Secretariat. > > > > > > _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg From bmwg-bounces@ietf.org Mon Jun 16 05:05:42 2008 Return-Path: X-Original-To: bmwg-archive@megatron.ietf.org Delivered-To: ietfarch-bmwg-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 33D623A690A; Mon, 16 Jun 2008 05:05:42 -0700 (PDT) X-Original-To: bmwg@core3.amsl.com Delivered-To: bmwg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AE94F3A690A for ; Mon, 16 Jun 2008 05:05:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8Ihz9q1gFJcg for ; Mon, 16 Jun 2008 05:05:34 -0700 (PDT) Received: from smtp-dub.microsoft.com (smtp-dub.microsoft.com [213.199.138.181]) by core3.amsl.com (Postfix) with ESMTP id 6B0B23A677C for ; Mon, 16 Jun 2008 05:05:34 -0700 (PDT) Received: from dub-exhub-c302.europe.corp.microsoft.com (65.53.213.92) by DUB-EXGWY-E802.partners.extranet.microsoft.com (10.251.129.2) with Microsoft SMTP Server (TLS) id 8.1.240.5; Mon, 16 Jun 2008 13:06:10 +0100 Received: from EA-EXMSG-C330.europe.corp.microsoft.com ([65.53.221.69]) by DUB-EXHUB-C302.europe.corp.microsoft.com ([65.53.213.92]) with mapi; Mon, 16 Jun 2008 13:06:10 +0100 From: Serge Ayoun To: Bill Cerveny , "bmwg@ietf.org" Date: Mon, 16 Jun 2008 13:06:08 +0100 Thread-Topic: [bmwg] Comments on draft-ietf-bmwg-protection-term-04.txt Thread-Index: AcjJxgbVwv8/GN4rRGGfWnfCGOZPcwF4xdxg Message-ID: <25817801063C9E418D1DE4B9DCF904C80CF6F0811E@EA-EXMSG-C330.europe.corp.microsoft.com> References: <9B4FD292-6805-414E-99AE-B47E9C4C04B6@wjcerveny.com> In-Reply-To: <9B4FD292-6805-414E-99AE-B47E9C4C04B6@wjcerveny.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 Cc: "jkarthik@cisco.com" , "sporetsky@nextpointnetworks.com" , Samir@core3.amsl.com, Rajiv Papneja , Vapiwala Subject: Re: [bmwg] Comments on draft-ietf-bmwg-protection-term-04.txt X-BeenThere: bmwg@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Benchmarking Methodology Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: bmwg-bounces@ietf.org Errors-To: bmwg-bounces@ietf.org Hi guys, My name is Serge and I am from Microsoft and have interest in performance and utm. I would like to participate and possibly have a look at your drafts. How can I find it if possible? Thanks, Serge A. -----Original Message----- From: bmwg-bounces@ietf.org [mailto:bmwg-bounces@ietf.org] On Behalf Of Bill Cerveny Sent: Monday, June 09, 2008 3:16 AM To: bmwg@ietf.org Cc: jkarthik@cisco.com; sporetsky@nextpointnetworks.com; Samir Vapiwala; Rajiv Papneja Subject: [bmwg] Comments on draft-ietf-bmwg-protection-term-04.txt My comments on draft-ietf-bmwg-protection-term-04.txt are below. I hope you find them useful and please feel free to contact me if you have any questions or need clarification. Bill Cerveny -------------------------------------- Overall, Bill's over-capitalization rant applies ;-) In general, there is way too much capitalization. Terms such as "working path", "protection," "backup path" or "path" should not be capitalized. In particular, look at IEEE reference on acronyms: http://www.computer.org/portal/pages/ieeecs/publications/author/style/acronyms.html Abstract: "The performance benchmarks are measured at the IP-Layer, so avoid dependence on specific sub-IP protection mechanisms." -- Consider rewording "... IP-Layer, so avoid" to something like "... IP-layer, avoiding..." or "... IP-layer and avoid ..." Introduction: (General comment ... is this too much detail for an introduction??) "Fast convergence times are critical to maintain reliable network connectivity and performance." - Consider changing to something like "Fast convergence times are critical for maintaining reliable network connectivity and performance." (Optional change) "Technologies that function at sub-IP layers can be enabled to provide further protection of IP traffic by providing the failure recovery at the sub-IP layers so that the outage is not observed at the IP-layer." - Reword to something like: "Technologies that provide failure recovery at the sub-IP layers can further protect IP traffic such that an outage is not observed at the IP layer." (Not perfect, but better, in my opinion) (Side observation: which "layer" standard are you referring to? Is this standard listed in references?) s/Benchmarking terminology have been defined/Benchmarking terminology has been defined/ "New terminology and methodologies specific to benchmarking sub-IP layer protection mechanisms are required. This will enable different implementations of the same protection mechanisms to be benchmarked and evaluated." - Perhaps change to "New terminology and methodologies specific to benchmarking sub-IP layer protection mechanisms will enable different implementations of the same protection mechanisms to be benchmarked and evaluated." New paragraph?: "The purpose of this document ..." New paragraph, changed text: "The sequence of events is:" "The Protection MAY..."?? Do you mean "Protection MAY ..." or "The protection system MAY..." Terminology Section Path: Why is L12 the first link and not something like L1? "The path is defined in the sub-IP layer in this document, unlike an IP path in RFC 2026 [1]." Perhaps change to "A path as defined in this document refers to a sub-IP layer path, unlike an IP path in RFC 2026 [1]." Backup Path: s/The Backup Path SHALL be the Working Path/The backup path SHALL become the working path/ Unclear - "A new Path computed after the Failover Event is simply Convergence [7] to a new Primary Path. " Disjoint Path: S/Discussions/Discussion/ :-) Point of Local Repair: s/Based on the functionality of the PLR, its role is defined based on the type of method used./The role of the PLR is based on the type of method used./ s/In the case the facility backup method is used/Where the facility backup method is used/ Link, Node and Path Protection definitions: To me, at least as they appear, "protection" appears to be a "condition" and not something physical such as a path (although they may describe the impact of a path). These definitions need to be defined as something like "condition where ..." or "state where ...". You may be able to convince me otherwise, but these definitions look initially awkward to me. Redundant Node Protection: Are VRRP and HA defined or referenced? Expand on acronym on first usage. Protected Interface: s/A Protected Interface is an interface protected by a Protection Switching Systems/A protected interface is an interface protected by a protection switching system/ (protection system system should be singular) Non-Protection System Node: S/A node that is not capable of participating in a Protection Switching System, however it MAY exist along the Primary Path or Backup Path./A node that is not capable of participating in a protection switching system. However, it MAY exist along the primary path or backup path./ Headend Node: Discussion begins and ends by discussing "headend" node, but in between, discussion is regarding "failover" nodes. What is relationship between headend and failover nodes? (Side comment: Acronym PLR is used infrequently enough that I have to think back to what the acronym refers to) Packet Loss Based Method: "Offered load rate" not defined, as far as I can tell. Additionally, the offered load rate measurement units aren't defined (which helps in understanding equation #3). Finally, the line "Units are ..." appears to state units are seconds, but "measurement units" are stated as milliseconds Time Based Method: Simplify, shorten or break-up: "The purpose of this method is to quantify the duration of failure or reversion on a time scale with granularity of milliseconds based on the observation of unimpaired packets, using Equation 2 with the difference being that the time values are obtained from the timestamp in the packet payload rather than from the Tester." _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg From bmwg-bounces@ietf.org Sat Jun 28 09:31:15 2008 Return-Path: X-Original-To: bmwg-archive-1@ietf.org Delivered-To: ietfarch-bmwg-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6487F3A67B1; Sat, 28 Jun 2008 09:31:15 -0700 (PDT) X-Original-To: bmwg@core3.amsl.com Delivered-To: bmwg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A58AC3A6843 for ; Sat, 28 Jun 2008 09:31:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.782 X-Spam-Level: X-Spam-Status: No, score=-104.782 tagged_above=-999 required=5 tests=[AWL=-0.444, BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.457, MSGID_FROM_MTA_HEADER=0.803, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pwAd+A1QbNfW for ; Sat, 28 Jun 2008 09:31:13 -0700 (PDT) Received: from mail146.messagelabs.com (mail146.messagelabs.com [216.82.245.3]) by core3.amsl.com (Postfix) with ESMTP id 84C933A67B1 for ; Sat, 28 Jun 2008 09:31:13 -0700 (PDT) X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-13.tower-146.messagelabs.com!1214670678!2924573!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.141] Received: (qmail 12590 invoked from network); 28 Jun 2008 16:31:19 -0000 Received: from sbcsmtp9.sbc.com (HELO flph161.enaf.ffdc.sbc.com) (144.160.128.141) by server-13.tower-146.messagelabs.com with AES256-SHA encrypted SMTP; 28 Jun 2008 16:31:19 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph161.enaf.ffdc.sbc.com (8.14.2/8.14.2) with ESMTP id m5SGVIKH002856 for ; Sat, 28 Jun 2008 09:31:18 -0700 Received: from klph001.kcdc.att.com (klph001.kcdc.att.com [135.188.3.11]) by flph161.enaf.ffdc.sbc.com (8.14.2/8.14.2) with ESMTP id m5SGV7O3002311 for ; Sat, 28 Jun 2008 09:31:12 -0700 Received: from kcdc.att.com (localhost.localdomain [127.0.0.1]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m5SGV7vx007019 for ; Sat, 28 Jun 2008 11:31:07 -0500 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m5SGV1Ia006994 for ; Sat, 28 Jun 2008 11:31:03 -0500 Message-Id: <200806281631.m5SGV1Ia006994@klph001.kcdc.att.com> Received: from acmt.att.com (unknown[135.210.112.87](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20080628163100gw100l7om9e>; Sat, 28 Jun 2008 16:31:01 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sat, 28 Jun 2008 12:30:58 -0400 To: Yaron Sheffer , Merike Kaeo , bmwg@ietf.org From: Al Morton In-Reply-To: <47E27594.1040903@checkpoint.com> References: <47E27594.1040903@checkpoint.com> Mime-Version: 1.0 Subject: Re: [bmwg] IPsec drafts - implementation issues X-BeenThere: bmwg@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Benchmarking Methodology Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1611926857==" Sender: bmwg-bounces@ietf.org Errors-To: bmwg-bounces@ietf.org --===============1611926857== Content-Type: text/html; charset="us-ascii" Hi Yaron, Merike, and all,

One of the chairman's jobs is to stimulate the discussion
needed to close open issues by summarizing a thread.
While most of the issues raised on the IPsec drafts were
discussed and closed, the message below is just hanging
out there.

*****************************************************************
I'm asking bmwg folks to think about this and weigh-in with their
opinions ASAP. The I-D submission deadlines are looming, and we
need to make some progress on this (and all our current work).
*****************************************************************

Yaron's message opened an issue, essentially pointing to
a mismatch between procedures which assume that the
negotiation phases are distinguishable/measurable, and
the capabilities of the equipment he and his colleagues are
familiar with. As I see it, we have some options:

A Move forward with the procedures as they are, endorsing
the useful information collected and giving the industry an
incentive to follow our recommendations.

B Modify the procedures to recognize limitations of some measurement
systems, possibly making the some steps optional (if that would work).

C+ Other options.

thanks for your attention,
Al
bmwg chair

At 10:32 AM 3/20/2008, Yaron Sheffer wrote:

Hi Merike,

Just when we thought we had finished this round...

I have been looking at the IPsec drafts with our performance lab people, and we have some questions regarding our ability to implement some of the tests.

Background:

For performance testing, we use COTS test equipment from the large equipment vendors. The equipment is IPsec-aware, but used in a black-box fashion. For example, you cannot pause between IKE phase 1 and phase 2 negotiation.

Specifics (all related to the methodology draft):

Tests 12.1, 12.2, 12.3 all require "single stepping" the different negotiation phases.
Similarly for 13.1 and 13.2.
Sec. 14 (this is NOT an implementation issue:) in addition to the many-tunnels case, we would also like to benchmark the single-tunnel, maximum throughput failover case. This is similar to many real life deployments of site-to-site VPNs.
Thanks,
Yaron

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg

--===============1611926857== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg --===============1611926857==-- From bmwg-bounces@ietf.org Mon Jun 30 04:44:05 2008 Return-Path: X-Original-To: bmwg-archive-1@ietf.org Delivered-To: ietfarch-bmwg-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BF8F03A69E1; Mon, 30 Jun 2008 04:44:05 -0700 (PDT) X-Original-To: bmwg@core3.amsl.com Delivered-To: bmwg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 96AB63A69E1 for ; Mon, 30 Jun 2008 04:44:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.437 X-Spam-Level: X-Spam-Status: No, score=-105.437 tagged_above=-999 required=5 tests=[AWL=0.359, BAYES_00=-2.599, MSGID_FROM_MTA_HEADER=0.803, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UtmuT7xO3r9p for ; Mon, 30 Jun 2008 04:44:03 -0700 (PDT) Received: from mail146.messagelabs.com (mail146.messagelabs.com [216.82.245.3]) by core3.amsl.com (Postfix) with ESMTP id 90E893A67AF for ; Mon, 30 Jun 2008 04:44:03 -0700 (PDT) X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-11.tower-146.messagelabs.com!1214826254!4224564!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.20.54] Received: (qmail 15955 invoked from network); 30 Jun 2008 11:44:14 -0000 Received: from sbcsmtp7.sbc.com (HELO mlpi135.enaf.sfdc.sbc.com) (144.160.20.54) by server-11.tower-146.messagelabs.com with AES256-SHA encrypted SMTP; 30 Jun 2008 11:44:14 -0000 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpi135.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id m5UBiEMZ013629 for ; Mon, 30 Jun 2008 07:44:14 -0400 Received: from alph001.aldc.att.com (alph001.aldc.att.com [135.53.7.26]) by mlpi135.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id m5UBi5RE013575 for ; Mon, 30 Jun 2008 07:44:09 -0400 Received: from aldc.att.com (localhost.localdomain [127.0.0.1]) by alph001.aldc.att.com (8.14.0/8.14.0) with ESMTP id m5UBi5ap027236 for ; Mon, 30 Jun 2008 07:44:05 -0400 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by alph001.aldc.att.com (8.14.0/8.14.0) with ESMTP id m5UBhwF8027162 for ; Mon, 30 Jun 2008 07:43:59 -0400 Message-Id: <200806301143.m5UBhwF8027162@alph001.aldc.att.com> Received: from acmt.att.com (unknown[135.210.32.209](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20080630114358gw100l7omle>; Mon, 30 Jun 2008 11:43:58 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 30 Jun 2008 07:43:58 -0400 To: bmwg@ietf.org From: Al Morton Mime-Version: 1.0 Subject: [bmwg] Preparing bmwg Agenda for IETF-72 X-BeenThere: bmwg@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Benchmarking Methodology Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: bmwg-bounces@ietf.org Errors-To: bmwg-bounces@ietf.org Folks, The main point of our face-to-face meetings is to have real interaction and reach agreements quickly, so that we accomplish our milestones on time... If you would like to request time on the bmwg agenda to *lead a discussion* on technical issues related to a working group draft, please respond to me with the topic and the description of technical issues. >>>> Please use the following format (this will help me greatly): TOPIC: Discussion GOALS: Draft file name, or preferably the complete URL: Presenter: >>>> IF none of the authors of a WG draft will be in Dublin, BMWG still needs the status/details for each work area!!! >>>> As always, I will try to make some meeting time available for proposed work at the end of the agenda. Let me know what proposals you would like to discuss, and how the bmwg can help with specific areas of draft development. I may have to do some triage on these requests, so that there is sufficient time to present and discuss some number of them (possibly less than the number requesting time). Please use the same form above. I need this info by Monday, July 7th, please. And, everyone can participate using remote means (jabber, Unicast Audio, and slides on the web) by reading the drafts and providing comments! I encourage you to try it if you can't join us in person. regards, Al bmwg chair _______________________________________________ bmwg mailing list bmwg@ietf.org https://www.ietf.org/mailman/listinfo/bmwg