Skip to content

Depth-too-short signal is blind to model-forward ⊤ values (no exposed call string) #733

Description

@khatchad

Summary

The depth-too-short signal added in #601 (PythonTensorAnalysisEngine.getDepthLimitedResults) reports an unknown (⊤) shape only when the value's node is a framework-API node whose call string is saturated to targetedCfaDepth. It is structurally blind to the ⊤ values that a deeper depth actually resolves, because those live on tf.keras.Model forward-method nodes, whose receiver-wrapped context (#679) exposes no call string.

Evidence

measureCallStringLength(context) reads CallStringContextSelector.CALL_STRING, which returns null for a TrampolineReceiverContextSelector-wrapped context. Model-forward nodes (Leaf.call, Dispatch.call) therefore report a call-string length of -1 and never satisfy length == targetedCfaDepth, so they are never reported.

Empirically, on the reproducer below the ⊤→concrete transition happens entirely on model-forward nodes at depth 0→1:

  • Depth 0: every model-forward call and both sinks are ⊤ ({? float32}); no framework node is ⊤.
  • Depth 1+: fully concrete ((3,) and (5,)).

So the depth that resolves the ⊤ is observable only on nodes the signal cannot measure. Conversely, the framework ⊤s the signal does report (e.g. neural_network.py's accuracy-path cast/reduce_mean/argmax/equal) are genuine ⊤s that do not resolve at a higher depth. The two categories do not overlap.

The receiver-instance context (#679) separates model-forward chains in a single call frame, so these transitions are consistently at depth 0→1, which the signal excludes by design (a length-0 context carries no call-string budget to deepen).

Reproducer

import tensorflow as tf


def consume(x):
    pass


class Leaf(tf.keras.Model):
    def call(self, x):
        return tf.einsum("ij->i", x)


class Dispatch(tf.keras.Model):
    def __init__(self, leaf):
        super().__init__()
        self.leaf = leaf

    def call(self, x):
        return self.leaf(x)


leaf = Leaf()
d = Dispatch(leaf)
a = d(tf.ones((3, 4)))
consume(a)
b = d(tf.ones((5, 6)))
consume(b)

Options

  1. Unwrap the receiver-wrapped context to read its inner call string, so model-forward nodes gain a measurable length, and reconsider whether a context-insensitive (length-0) targeted ⊤ should be reported. Reporting length-0 ⊤s makes the signal fire on the resolvable model-forward ⊤s but broadens over-reporting at depth 0 (every targeted ⊤ becomes a candidate), so it is a semantics change to weigh, not a drop-in fix.
  2. Keep the signal scoped to framework-node truncation and document that model-forward depth-0 ⊤s are out of scope.

This is a known-gap filing, not a request to change core behavior under time pressure. It is relevant to #670 (surfacing the signal in the evaluator): a consumer should know the signal covers framework truncation, not the model-forward depth-0 case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions