Python Eager Execution

Relevant source files

This document covers TensorFlow's eager execution system in Python, which allows operations to be executed immediately without building a computational graph. It details the EagerTensor Python bindings, NumPy integration, data conversion pipeline, and the mechanisms for executing TensorFlow operations in eager mode.

For information about Go language bindings, see page 4.2. For cross-language data conversion details, see page 4.3. For tf.function graph tracing that builds on eager execution, see page 9.

Eager Execution Architecture

TensorFlow's eager execution allows operations to execute immediately, returning concrete values instead of building a computational graph. The system consists of Python bindings, C++ execution infrastructure, and data conversion mechanisms.

Eager Execution Data Flow

Sources: tensorflow/python/eager/pywrap_tensor.cc1-100 tensorflow/python/eager/pywrap_tfe_src.cc120-150 tensorflow/core/common_runtime/eager/execute.cc150-200 tensorflow/python/eager/context.py1-100

EagerTensor Python Bindings

The EagerTensor type is the Python representation of a tensor during eager execution. It is implemented as a C extension for performance.

EagerTensor Structure

The EagerTensor C++ struct provides the Python-facing representation. It bridges the Python PyObject world with the C++ TFE_TensorHandle.

Field	Type	Purpose
`handle`	`TFE_TensorHandle*`	Pointer to the underlying C API tensor handle.
`context`	`TFE_Context*`	The eager context in which this tensor lives.
`id`	`int64_t`	Unique identifier for the tensor, used for caching and tracing.

Sources: tensorflow/python/eager/pywrap_tensor.h1-50 tensorflow/python/eager/pywrap_tensor.cc1-100

EagerTensor Object Lifecycle

Sources: tensorflow/python/eager/pywrap_tensor.cc1-200 tensorflow/core/common_runtime/eager/context.cc1-100 tensorflow/c/eager/c_api.cc1-100

Operation Execution

Executing an operation in eager mode involves the quick_execute function, which dispatches the operation to the C++ runtime.

Execution Pipeline

When a Python op (e.g., tf.add) is called in eager mode:

It calls execute.quick_execute in tensorflow/python/eager/execute.py28-67
This invokes the pybind11-wrapped TFE_Py_Execute in tensorflow/python/eager/pywrap_tfe_src.cc1-100
The C++ runtime selects the appropriate kernel and executes it via TFE_Execute in tensorflow/core/common_runtime/eager/execute.cc135-170

Callbacks and Monitoring

Eager execution supports callbacks for debugging and profiling. execute_with_callbacks allows intercepting op execution. Monitoring gauges, such as those in tensorflow/python/framework/ops.py104-115 track API usage.

Sources: tensorflow/python/eager/execute.py28-67 tensorflow/python/eager/pywrap_tfe_src.cc1-100 tensorflow/core/common_runtime/eager/execute.cc135-170 tensorflow/python/framework/ops.py104-115

NumPy Integration and Data Conversion

TensorFlow provides tight integration with NumPy, allowing EagerTensor to be used in place of NumPy arrays and vice versa.

NumPy to Tensor (Zero-Copy)

The conversion pipeline attempts to share memory between NumPy and TensorFlow whenever possible.

NdarrayToTensor in ndarray_tensor.cc handles the buffer sharing logic.
The EagerTensor object implements the __array__ protocol to allow NumPy to consume it.

Data Conversion Pipeline

For non-NumPy types (like Python lists), the system uses PySeqToTFE_TensorHandle in py_seq_tensor.cc to infer shapes and types recursively.

Data Conversion Pipeline

Sources: tensorflow/python/lib/core/ndarray_tensor.cc1-100 tensorflow/python/lib/core/py_seq_tensor.cc1-100 tensorflow/python/eager/pywrap_tensor_conversion.cc1-50

Eager Context Management

The Context class in tensorflow/python/eager/context.py218-300 manages the global state for eager execution, including device placement policies, executor configuration (sync vs. async), and function libraries.

Key Context Settings

Device Placement: Controls how ops are assigned to devices (e.g., DEVICE_PLACEMENT_EXPLICIT in tensorflow/python/eager/context.py69-74).
Execution Mode: Toggles between synchronous and asynchronous execution using SYNC and ASYNC constants tensorflow/python/eager/context.py76-77
JIT Compilation: Enables or disables XLA JIT compilation for eager ops tensorflow/python/eager/context.py104-127

Sources: tensorflow/python/eager/context.py218-300 tensorflow/python/eager/context.py69-77 tensorflow/python/eager/context.py104-127

tf.function and Tracing

While eager execution is the default, tf.function allows for performance optimizations by tracing Python code into a graph.

Tracing: When a tf.function is first called, it runs in eager mode but records operations into a FuncGraph.
ConcreteFunction: The result of tracing is a ConcreteFunction tensorflow/python/eager/function.py26
AtomicFunction: Low-level representation of a function that can be executed by the C++ runtime tensorflow/python/eager/function.py28

Sources: tensorflow/python/eager/function.py21-38 tensorflow/python/framework/ops.py112-115