Miscellaneous

Checkpoint related

• Comparison:

  • Checkpoint(*.ckpt)

    • Contains three files normally: model.ckpt.index model.ckpt.meta model.ckpt.data-00000-of-00001

    • Saved by tf.trainer.Saver().save(), only save the tf.Variables(), not including graph model, thus impossible to recover a whole graph with ckpt only.

    • Use saver.restore(session, checkpoint_path) to restore.

  • GraphDef(*.pb)

    • Contains serialized protobuf data and compute graph, but not Variable data.

    • Can only recover compute graph, need checkpoint to inject data.

    • FrozenGraphDef is slightly different, since it will convert all Variable to constant values(which can be loaded from checkpoint), usually used as pre-train model.

  • SavedModel

    • Combination of checkpoint and GraphDef, plus SignatureDef of input and output variables.

Reference: TensorFlow 到底有几种模型格式？

checkpoint

• Print parameters in checkpoint

1. Using internal tool -- inspect_checkpoint:

   python -m tensorflow.python.tools.inspect_checkpoint --file_name=model.ckpt "$@"

2. Using tf.compat.v1.NewCheckpointReader(ckpt_path):

   ckpt_path = "./model.ckpt"
   reader = tf.compat.v1.train.NewCheckpointReader(ckpt_path)
   var_to_shape_map = reader.get_variable_to_shape_map()
   for key in var_to_shape_map:
       print(f"{key}: {reader.get_tensor(key)}")

Reference: 如何打印出TensorFlow保存的checkpoint里的参数名

saved_model

pb

keras h5

Keras save its model in h5 format, so we need to know how to read h5 in python if we only want to search for some specific parameters.

# import related lib
import h5py

# init a file object with only-read permission
f = h5py.File(path_to_h5, 'r')

# list its keys
list(f.keys())

# h5 file can be indexed by keywords
list(f['model_weights'].keys()) 

# indexed into deeper layer
f['model_weights']['dense_73']
# Out: <HDF5 group "/model_weights/dense_73" (1 members)>
# Group means it has children leaves, can be indexed deeper by key

f['model_weights']['dense_73']['dense_73']['kernel:0']
# Out: <HDF5 dataset "kernel:0": shape (768, 22), type "<f4">
# dataset means it can be read now

# dataset can be read as list, numpy, etc.
list(f['model_weights']['dense_73']['dense_73']['bias:0'])

# dataset also has shape
f['model_weights']['dense_73']['dense_73']['kernel:0'].shape
# Out: (768, 22)

Reference: 模型复现之HDF5文件