This presentation was given at the Network Management Area:
Open Meeting on Tuesday, March 29, 1994.


SNMP Test Summit -- Conformance and Interoperability Testing:
Experience and Future Plans

Presentation by Chris Wellens
InterWorking Labs, Inc.

Goals:

.Provide neutral, independent, unbiased testing.
.Provide the highest quality test suites and tools.
.Provide a forum to discuss outstanding technical issues and
 resolution of ambiguities.
.Build on existing test suite base.

=============================================================

First SNMP Test Summit
San Jose, CA
Jan. 10 - 15, 1994

Participating Companies:

Cabletron Systems, Inc.
cisco Systems, Inc.
Eicon Technology
Empirical Tools and Technolgies
Epilogue Technology Corp.
Fujitisu OSSI
IBM and IBM Research
Network General Corp.
PEER Networks
SNMP Research
SynOptics Communications
TGV, Inc.
Wellfleet Communications

==============================================================

Announcing the Second SNMP Test Sumit

Holiday Inn Park enter Plaza
282 Almaden Blvd.
San Jose, CA

June 27 - July 1, 1994

New Areas for Testing:

.Instrumentation (MIB II, RMON, traps)
.SNMPv2
.Additional manager tests
.Bug fixes (send email to:  tsbugs-l@netcom.com)
.Expanded documentation

Participants will receive SNMP Test Suite v.2.0 at the end of
the Summit!

==============================================================

Planned Enhancements to the Tests


1.  Add instrumentation testing for MIBII, RMON, and traps.
2.  Constrain walk on 1.1.2.  Save MIB walk information before
    exiting.  
3.  Store output of walk in a file that is editable.
4.  Provide additional support for subagent testing.
    (Ability to specify objects when doing operation on a mix of
    objects from different subagents.)
5.  Sets on multiple objects with ability to specify objects.
6.  Add optimization to tests.  Do tests on only one row of a
    table, not all.
7.  Add weighted reporting, e.g. getnext failures are worse than set
    failures.
8.  Check if the agent is still alive in the case where timeout
    is a valid expected response.
9.  In case of "dynamic tables", the error reporting for
    retrieval failures should include information on which variables
    failed.
10. More descriptions and information on expected outcomes.
11. Once a problem has been found, no need to perform same test
    on multiple instances.
12. Ability to stop ongoing tests.
13. Add more manager testing.  Consider designing the manager
    tests according to different application areas that are
    representative of the kinds of things customers try to do.
14. Need more explanation on the point of the test.

=========================================================

Complete Test Specification Available April 18, 1994

Use Anonymous ftp to netcom.com
cd ~ftp/pub/snmp-test
get june.test.spec

Call or email questions and comments to:
chrisw@netcom.com
tel 415-969-4544