tf.VariableSynchronization

Indicates when a distributed variable will be synced.

  • AUTO: Indicates that the synchronization will be determined by the current DistributionStrategy (eg. With MirroredStrategy this would be ON_WRITE).
  • NONE: Indicates that there will only be one copy of the variable, so there is no need to sync.
  • ON_WRITE: Indicates that the variable will be updated across devices every time it is written.
  • ON_READ: Indicates that the variable will be aggregated across devices when it is read (eg. when checkpointing or when evaluating an op that uses the variable).
Class Variables
AUTO tf.VariableSynchronization
NONE tf.VariableSynchronization
ON_READ tf.VariableSynchronization
ON_WRITE tf.VariableSynchronization

© 2020 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/versions/r2.4/api_docs/python/tf/VariableSynchronization