Hi,

I have been selected as the Operational Directorate (opsdir) reviewer for this Internet-Draft.

The Operational Directorate reviews all operational and management-related Internet-Drafts to ensure alignment with operational best practices and that adequate operational considerations are covered.

A complete set of _"Guidelines for Considering Operations and Management in IETF Specifications"_ can be found at https://datatracker.ietf.org/doc/draft-ietf-opsawg-rfc5706bis/.

While these comments are primarily for the Operations and Management Area Directors (Ops ADs), the authors should consider them alongside other feedback received.

- Document: draft-ietf-cats-metric-definition-10
- Reviewer: Fung Lim
- Review Date: 3rd July 2026
- Intended Status: Standards Track

---

## Summary

- Has Issues: I have concerns about this document that I think should be resolved before publication.

This document defines a three-level hierarchical abstraction for Computing-Aware Traffic Steering (CATS) metrics, standardizing Level 1 category metrics (computing, communication, service, composed) and a Level 2 global normalized score, along with a metric field framework. The design is well-motivated: it deliberately avoids standardizing fine-grained Level 0 metrics while providing interoperable encodings and normative score semantics at higher levels. The metric framework described in this document is sound and several operational concerns are implicitly addressed.

However, as a Standards Track document that defines inputs to traffic-steering decisions, it requires a dedicated Operational Considerations section (per Section 3.1 of draft-ietf-opsawg-rfc5706bis). Several operational topics that are partially scattered across design principles, registry entries, and security text would benefit from consolidation into operator-facing guidance. In particular, cross-vendor score comparability, metric collection failure handling, control-plane churn from metric updates, and management interface expectations should be addressed.


1. Missing Operational Considerations section (§3.1 of rfc5706bis)

Per draft-ietf-opsawg-rfc5706bis, a technical specification advancing as an IETF RFC MUST include an Operational Considerations section immediately before Security Considerations, or explicit boilerplate with rationale if none apply. This document jumps from Section 6 (CATS Metric Registry Entries) directly to Section 7 (Security Considerations).

For a Standards Track document whose metrics directly influence instance selection and traffic steering, an Operational Considerations section is essential rather than optional. The document already contains operationally relevant material — score semantics (Section 4.3), level selection trade-offs (Section 5), role assignments (C-SMA, C-NMA, C-PS in Section 6), calibration guidance, and security requirements — but it is not consolidated where operators will look for it.


2. Cross-vendor score comparability needs explicit operational guidance (§4.1, §4.5 of rfc5706bis)

Section 4.3 warns that normalized scores "may not be directly comparable across different implementations" because aggregation and normalization functions are implementation-specific. Section 4.2.1 and 4.2.2 state these functions "are not standardized" and are "controlled by operator policies."

This is a fundamental operational constraint that affects multi-vendor and multi-provider CATS deployments. An operator comparing a score of 7 from Provider A against a score of 7 from Provider B has no assurance of equivalent capability. The normative 0–3 / 4–7 / 8–10 steering bands (Section 4.3) help within a single administrative domain but do not solve cross-domain comparison.

The Operational Considerations section should address:

a. When scores are safe to use for steering (intra-domain only vs. inter-domain)
b. Whether operators should prefer Level 1 category metrics over Level 2 global scores for multi-vendor environments
c. How calibration (referenced in Section 6 registry entries) should be applied operationally — who runs benchmarks, how often, and what deviation thresholds trigger re-calibration
d. Whether future CATS signaling documents will carry normalization policy identifiers to aid comparison


3. No fault management or troubleshooting guidance (§5.4 of rfc5706bis)

CATS metrics drive steering decisions. The document does not discuss how operators detect or respond to metric-related faults:

Stale or missing metrics from C-SMA or C-NMA (Section 7 addresses freshness at the security level via SEC-4, but not operational fallback behavior)
Collection agent failures or telemetry pipeline outages
Score anomalies (sudden drops, stuck values, contradictory Level 1 vs. Level 2 scores)
Integrity violation handling (SEC-1 says metrics SHOULD trigger alerts, but no alert schema or operational procedure)
Per §5.4 of rfc5706bis, essential faults, alarms, and events should be identified. At minimum, the document should discuss:

a. Fallback steering behavior when metrics are unavailable or fail freshness checks
b. Whether last-known-good values are used or instances are de-prioritized
c. Recommended health indicators for C-SMA, C-NMA, and C-PS components



4. No guidance on verifying correct operation (§4.7 of rfc5706bis)

The registry entries include calibration methods (e.g., "score deviation ... less than 0.1 over repeated test rounds") but the document does not describe how operators verify end-to-end CATS metric operation in production:

Active vs. passive validation approaches
Synthetic workload injection for score validation
Correlation between normalized scores and observed application QoE
Simple health checks (e.g., confirming C-SMA is publishing expected metric types at expected intervals)
Section 6 registry entries reference benchmark calibration but treat it as an implementation activity without operator-facing procedures.



5. Metric update frequency and control-plane impact not addressed (§4.5 of rfc5706bis)

Section 3.1 discusses the trade-off between metric update frequency and "signaling overhead, reducing scalability of the metric distribution protocol." Section 6 suggests continuous sampling every 10 seconds with a default Measurement_Window of 10 seconds.

However, the document does not quantify or guide:

Recommended maximum metric advertisement rate per service instance
Impact of high churn on CATS control-plane protocols (to be defined in companion documents)
When coarser sampling or Level 2-only advertisement is preferable to reduce overhead
Interaction between Statistics field choices (max, min, mean, cur) and update frequency
This is particularly important because CATS may scale to hundreds or thousands of service instances (Section 5).


Nits

6. No management interoperability discussion (§5.2 of rfc5706bis)

The document references Prometheus, NETCONF, and IPFIX for Level 0 metric collection but does not discuss:

Whether a YANG data model for CATS metric configuration, policy, and exposed state is needed or planned
How normalized scores and runtime parameters (Measurement_Window, aggregation weights) would be configured via standard management interfaces
Distinction between configuration data (normalization bounds, weights), operational state (current scores), and statistics (historical trends)
For a Standards Track metric definition consumed by network elements, some discussion of management model expectations — even if deferred to a companion document — would help operators plan deployments.


7. Migration and coexistence with existing metrics (§4.3 of rfc5706bis)

The Introduction references existing TE metrics (RFC 7471) and network performance registries (RFC 8911, RFC 8912). The document does not discuss:

How CATS Level 1 communication metrics relate to or subsume existing IGP TE metrics
Whether operators can incrementally adopt CATS metrics alongside traditional TE without dual-metric steering conflicts
Mapping or equivalence between ALTO performance cost metrics (RFC 9439) and CATS normalized scores
A brief migration narrative would help operators who already run TE metric extensions.


8. Role assignment ambiguity in registry entries

Section 6.4.3.6 assigns metric calculation to "Service contact instance" rather than C-SMA (used consistently elsewhere). Section 6.1.3.6 contains a formatting error where ### Output appears inline after the C-PS role description, disrupting the registry structure.

Section 6.4.3.6:

Service contact instace: Collects Level 0 service raw metrics...

This should likely reference C-SMA for consistency with the framework terminology.


9. Editorial and reference nits


"Section Section 4.4" (Section 4.1, line 605) — duplicate word
"To-be-assgined" → "To-be-assigned" (Sections 6.1.4.2, 6.2.5.2, 6.3.5.2)
"Service contact instace" → "Service contact instance" (Section 6.4.3.6)
Normative self-reference [I-D.ietf-cats-metric-definition] points to draft-08 (Section 9.1) — should reference the current draft version
Appendix A examples use inconsistent Source capitalization ("Directly measured" vs. lowercase 'directly measured' in Section 4.1)
IANA placeholder "XXX" identifiers throughout Section 6 — expected for draft stage, but Operational Considerations should note operators need stable identifiers before deployment


10. Accounting and performance management partially addressed

Section 6 registry entries define singleton output values and calibration targets, which is a good start. However:

No counter rollover behavior for Measurement_Window accumulators
No guidance on historical metric retention for capacity planning
No discussion of metric export for billing or SLA reporting (may be N/A for some deployments, but worth a sentence)

Recommendations

1. Add an Operational Considerations section immediately before Security Considerations. Consolidate scattered operational material and address: metric level selection, runtime parameter defaults, cross-vendor score limitations, metric collection failures and fallback steering, update rate guidance, and coexistence with existing TE metrics.

2. Document fault management and fallback behavior. Specify what happens when C-SMA/C-NMA stops publishing, metrics fail freshness checks (SEC-4), or scores are contradictory. Define minimum health indicators and recommended alert conditions.

3. Add verification guidance. Describe how operators validate that normalized scores reflect actual instance capability — referencing the calibration methods already in Section 6 registry entries and extending them into deployable procedures (synthetic workloads, periodic re-calibration, QoE correlation).

4. Clarify cross-domain and multi-vendor deployment constraints. Expand Section 4.3's comparability warning into explicit operational guidance: when scores are trustworthy, when Level 1 categories are preferable to Level 2 global scores, and how calibration maintains consistency within an administrative domain.

5. Discuss management model expectations. Even if a YANG module is deferred, state whether configuration of normalization bounds, aggregation weights, and Measurement_Window is expected via NETCONF/RESTCONF or operator-local policy, and how config vs. operational state vs. statistics should be distinguished.

6. Address metric advertisement churn. Provide guidance on sampling intervals, maximum update rates, and when to prefer Level 2 over Level 1 to limit control-plane overhead at scale.

7. Fix editorial issues in Section 6 registry entries (inline ### Output, role naming inconsistency, typos) and update the normative self-reference to the current draft version.

8. Add a brief migration narrative explaining how CATS metrics complement or replace existing IGP TE metrics and ALTO cost metrics during incremental adoption.

I hope these comments are useful and constructive. The hierarchical metric framework is well-conceived and addresses a genuine interoperability need for CATS; adding consolidated operational guidance will significantly improve its deployability in multi-vendor environments.

Lim Fung